Pneumonia detection has important research significance in medical image processing. For the problem that the current classical detection algorithms has low accuracy in detecting pneumonia lesions. This paper presents an algorithm for detecting pneumonia lesions in X-ray images based on multi-scale convolutional neural networks. The feature channel attention module is added to the basic feature extraction network to highlight the channel containing useful information in the feature map, and to suppress the feature channel without lesion information or containing a large amount of useless information to form a high-quality feature map. Then through statistical analysis, a series of candidate frames with different aspect ratios and scaling scales are designed using clustering algorithm to be suitable for pneumonia lesion detection. In this paper, the single-model and multi-model detection experiments are performed on chest X-ray datasets containing pneumonia lesions. The detection accuracy is 82.52% in the case of single model and 89.08% in the case of multi-model fusion. Through comparison experiments and results analysis, the proposed algorithm is superior to other detection algorithms in pneumonia lesion detection and is suitable for pneumonia lesion detection in X-ray images.