Super-resolution (SR) is a widely used technology that increases image resolution using algorithmic methods. However, preserving the local edge structure and visual quality in infrared (IR) SR images is challenging because of their disadvantages, such as lack of detail, poor contrast, and blurry edges. Traditional and advanced methods maintain the quantitative measures, but they mostly fail to preserve edge and visual quality. This paper proposes an algorithm based on high frequency layer features. This algorithm focuses on the IR image edge texture in the reconstruction process. Experimental results show that the mean gradient of the IR image reconstructed by the proposed algorithm increased by 1.5, 1.4, and 1.2 times than that of the traditional algorithm based on L1- norm, L2-norm, and traditional mixed norm, respectively. The peak signal-to-noise ratio, structural similarity index, and visual effect of the reconstructed image also improved.