In order to improve the resolution of low-resolution infrared images, this paper proposes an infrared image super-resolution algorithm based on random forest. Firstly, two random forests models are trained independently. The first model is trained by using infrared images, while the second is trained by using registered multi-sensor images. Then, an adaptive extraction algorithm is utilized to extract the edges of infrared images and registered visible light images. The correlation coefficient between the low-resolution patch of infrared image and the high-resolution patch of visible light image is calculated. According to the correlation coefficient, an appropriate model can be selected. Finally, the selected model is utilized to reconstruct the high-resolution infrared patch. All these patches are integrated into a high-resolution infrared image. The experimental results show that the proposed method can obtain better performance compared with the super-resolution random forest algorithm. The Peak Signal to Noise Ratio(PSNR) of testing images is increased by 0.09 dB on average. The reconstructed images, with better visual effect, are closer to original images.