Image super-resolution algorithms based on convolution neural network can be classified into two steps: image size enlargement and image detail recovery/enhancement. During the detail recovery process, the convolution layer learns the feature directly from the input image and takes the feature as the input data of the next convolution layer. In this study, a novel convolution neural network algorithm is proposed to enhance the feature expression ability of input and channel images in convolution layers by the selective gray transformation of the input and channel images. The experimental results demonstrate that the super-resolution reconstruction effect of the proposed method is superior to several typical algorithms in both conventional infrared images and the infrared images collected from our laboratory, and the proposed method can be applied to recover more details.