Infrared and Laser Engineering, Volume. 53, Issue 5, 20240049(2024)

Deep learning-based infrared imaging degradation model identification and super-resolution reconstruction

Junfeng Cao1,2,3,4, Qinghai Ding5, Depeng Zou6, Hengjia Qin6, and Haibo Luo1,2,3
Author Affiliations
  • 1Key Laboratory of Opto-Electronic Information Processing, Chinese Academy of Sciences, Shenyang 110016, China
  • 2Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, China
  • 3Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang 110169, China
  • 4University of Chinese Academy of Sciences, Beijing 100049, China
  • 5Space Star Technology Co., Ltd., Beijing 100086, China
  • 6The Third Military Representative Office of the Air Force Equipment Department in Shenyang, Shenyang 110016, China
  • show less

    ObjectiveThe limited resolution of infrared devices, constrained by cost and manufacturing technology, remains a challenge. While deep learning-based single image super-resolution (SISR) has shown promise in enhancing image resolution, its application in real-world infrared images is hindered by the complexity of actual degradation, including spatial non-uniform blur caused by optical aberration and assembly error, as well as variations in the blur kernel due to environmental temperature changes. A deep learning-based approach for infrared imaging degradation model identification and super-resolution reconstruction is proposed to tackle these challenges. This method entails solving the degradation model using a convolutional neural network to describe the evolution of blur kernels, along with a super-resolution reconstruction method that adheres to the constraints of the degradation model and incorporates online learning of degradation parameters.MethodsImages of calibration targets are captured using an infrared camera placed in a high and low temperature chamber, along with a portable target simulator placed outside it (Fig.1-2). These images are utilized to calibrate the blur kernels. A convolutional neural network (CNN) is employed to construct a model that characterizes the relationship between blur kernel, pixel coordinate, and operating temperature (Fig.3). The model is trained using the calibrated blur kernels. Additionally, a super-resolution network is developed and trained (Fig.4). The operating temperature is initially estimated using the low-resolution image. Next, the initial blur kernels are estimated by inputting the operating temperature into the kernel model. Subsequently, super-resolution reconstruction is conducted based on the estimated blur kernels, and the reconstructed image is utilized to refine the operating temperature and blur kernel estimation. Iterative processes improve the accuracy of blur kernel estimation, leading to enhanced reconstruction outcomes.Results and DiscussionsThe blur kernels of the infrared imaging system exhibit significant variation in response to temperature changes and spatial locations (Fig.6). The trained blur kernel model effectively predicts blur kernels using temperature and pixel coordinate inputs (Fig.7). The average PSNR between predicted and actual blur kernels across different operational temperatures is consistently high, with a minimum of 32.2 dB and an average of 37.1 dB, indicating precise predictions (Fig.8). The calibration and modeling of blur kernels provide valuable prior information for super-resolution reconstruction, resulting in enhanced reconstruction outcomes. Consequently, the proposed algorithm produces visually appealing results with improved detail (Fig.10-11) and enhances objective quality evaluation metrics such as the natural image quality evaluator (NIQE), perception-based image quality evaluator (PIQE), and blind/referenceless image spatial quality evaluator (BRISQUE) (Tab.1).ConclusionsA novel approach is proposed for infrared super-resolution imaging, including degradation model identification and iterative super-resolution reconstruction. The degradation model is based on a convolutional neural network and is solved using offline calibration data. It can predict blur kernels across various temperatures and spatial positions, reducing the need for extensive calibration work. Online degradation parameter correction is achieved through an iterative optimization network alternating between estimating the blur kernel and reconstructing the super-resolution image. By leveraging the degradation model, the complex high-dimensional blur kernel estimation problem is simplified into a low-dimensional operating temperature estimation problem, streamlining the solution process. Through iterations, the accuracy of blur kernel estimation improves, leading to superior super-resolution reconstruction outcomes. Experimental results demonstrate that calibrating and modeling blur kernels enhance prior information for super-resolution reconstruction, yielding superior results. Additionally, the proposed method adapts to a wider temperature range, reducing the stringency of athermalization design requirements for infrared optical systems.

    Keywords
    Tools

    Get Citation

    Copy Citation Text

    Junfeng Cao, Qinghai Ding, Depeng Zou, Hengjia Qin, Haibo Luo. Deep learning-based infrared imaging degradation model identification and super-resolution reconstruction[J]. Infrared and Laser Engineering, 2024, 53(5): 20240049

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category:

    Received: Jan. 27, 2024

    Accepted: --

    Published Online: Jun. 21, 2024

    The Author Email:

    DOI:10.3788/IRLA20240049

    Topics