A super-resolution reconstruction method of infrared images based on channel attention and transfer learning was proposed to solve the problems of low resolution and low quality of infrared images. In this method, a deep convolutional neural network is designed to enhance the learning ability of the network by introducing the channel attention mechanism, and the residual learning method is used to mitigate the problem of gradient explosion or disappearance and to accelerate the convergence of the network. Because high-quality infrared images are difficult to collect and insufficient in number, so this method is divided into two steps: the first step is to use natural images to pre-train the neural network model, and the second step is to use transfer learning knowledge to fine-tune the pre-trained model’s parameters with a small number of high-quality infrared images to make the model better in reconstructing the infrared image. Finally, a multi-scale detail boosting filter is added to improve the visual effect of the reconstructed infrared image. Experiments on Set5 and Set14 datasets as well as infrared images show that the deepening network depth and introducing channel attention mechanism can improve the effect of super-resolution reconstruction, transfer learning can well solve the problem of insufficient number of infrared image samples, and multi-scale detail boosting filter can improve the details and increase the amount of information of the reconstruction image.