Journals Articles Conferences News About CLP
Submit a paper Email Alert
Search
Search
Articles
Journals
News
Advanced Search
Top Searches
2D MaterialsTransformation opticsQuantum PhotonicsSmall lasersSemiconductor UV PhotonicsSupersymmetric microring laser arrays

Acta Optica Sinica, Volume. 45, Issue 17, 1720017(2025)

Review of Infrared Image Colorization Technology (Invited)

Xiubao Sui1、*, Yuan Liu1, Tong Jiang1, Tingting Liu1, and Qian Chen1,2
Author Affiliations
  • 1School of Electronic and Optical Engineering, Nanjing University of Science and Technology, Nanjing 210094, Jiangsu , China
  • 2School of Information and Communication Engineering, North University of China, Taiyuan 030051, Shanxi , China
    • show less
    AbstractGet PDF(in Chinese)
    Figures&Tables (22)
    Equations (0)
    References (53)
    Tools
    Paper Information
    Topics
    Figures & Tables(22)
    Classification of infrared image colorization methods

    Fig. 1. Classification of infrared image colorization methods

    Download full size
    Principle block diagram based on preset mapping strategy

    Fig. 2. Principle block diagram based on preset mapping strategy

    Download full size
    Infrared effect based on preset mapping strategy. (a) Infrared image; (b) pseudo-color image

    Fig. 3. Infrared effect based on preset mapping strategy. (a) Infrared image; (b) pseudo-color image

    Download full size
    Principle diagram of color transfer method based on global matching

    Fig. 4. Principle diagram of color transfer method based on global matching

    Download full size
    Principle diagram of color transfer method based on pixel matching

    Fig. 5. Principle diagram of color transfer method based on pixel matching

    Download full size
    Colorization effect of infrared image based on color transfer. (a)(b) Input image; (c)(d) reference image; (e)(f) colorization result

    Fig. 6. Colorization effect of infrared image based on color transfer. (a)(b) Input image; (c)(d) reference image; (e)(f) colorization result

    Download full size
    Colorization effect of infrared image based on multi-spectral fusion. (a) Input image; (b) reference image; (c) output image

    Fig. 7. Colorization effect of infrared image based on multi-spectral fusion. (a) Input image; (b) reference image; (c) output image

    Download full size
    Overview of infrared image colorization methods based on deep learning

    Fig. 8. Overview of infrared image colorization methods based on deep learning

    Download full size
    Several typical infrared image colorization network structures and learning strategies. (a) CNN-based infrared image colorization strategy; (b) supervised GAN-based infrared image colorization strategy (e.g., Pix2pix[27]); (c) one-way unpaired GAN-based contrastive loss strategy (e.g., CUT[38]); (d) one-way unpaired GAN-based attention-guided contrastive loss strategy (e.g., OS-Attn[41]); (e) two-way unpaired GAN-based cycle consistency loss strategy (e.g., CycleGAN[33]); (f) two-way unpaired GAN-based contrastive loss strategy (e.g., DCLGAN[43])

    Fig. 9. Several typical infrared image colorization network structures and learning strategies. (a) CNN-based infrared image colorization strategy; (b) supervised GAN-based infrared image colorization strategy (e.g., Pix2pix[27]); (c) one-way unpaired GAN-based contrastive loss strategy (e.g., CUT[38]); (d) one-way unpaired GAN-based attention-guided contrastive loss strategy (e.g., OS-Attn[41]); (e) two-way unpaired GAN-based cycle consistency loss strategy (e.g., CycleGAN[33]); (f) two-way unpaired GAN-based contrastive loss strategy (e.g., DCLGAN[43])

    Download full size
    Generative adversarial network model

    Fig. 10. Generative adversarial network model

    Download full size
    Network architecture of CycleGAN

    Fig. 11. Network architecture of CycleGAN

    Download full size
    Basic framework of contrastive learning

    Fig. 12. Basic framework of contrastive learning

    Download full size
    Typical infrared image colorization network effect diagram

    Fig. 13. Typical infrared image colorization network effect diagram

    Download full size
    Detection results of YOLOv7 on colorized images. (a) Input nighttime infrared image; (b) nighttime RGB image; (c) FRAGAN result; (d) LKAT-GAN result

    Fig. 14. Detection results of YOLOv7 on colorized images. (a) Input nighttime infrared image; (b) nighttime RGB image; (c) FRAGAN result; (d) LKAT-GAN result

    Download full size
    Principle framework of TeX-Net[50]

    Fig. 15. Principle framework of TeX-Net[50]

    Download full size
    Physically driven colorization of infrared hyperspectral image based on HADAR. (a)(b)(e)(f) Input image; (c)(d)(g)(h) colorized image

    Fig. 16. Physically driven colorization of infrared hyperspectral image based on HADAR. (a)(b)(e)(f) Input image; (c)(d)(g)(h) colorized image

    Download full size
    High frequency similarity between infrared image and visible image. (a) Visible image; (b) infrared image; (c) high-frequency information of visible image; (d) high-frequency information of infrared image

    Fig. 17. High frequency similarity between infrared image and visible image. (a) Visible image; (b) infrared image; (c) high-frequency information of visible image; (d) high-frequency information of infrared image

    Download full size
    Zero-shot cross-modal colorization method framework[51]

    Fig. 18. Zero-shot cross-modal colorization method framework[51]

    Download full size
    Colorization effect of infrared image based on zero-shot learning[51]

    Fig. 19. Colorization effect of infrared image based on zero-shot learning[51]

    Download full size

    • Table 0. [in Chinese]

      View table

      Table 0. [in Chinese]

      TypeMethodNetwork

      Learning

      strategy

      		StrengthWeakness
      GANCTSC[37]● Uses CUT architectureUnsupervised● Introduces topology-aware GNN and attention module

      ● Requires graph construction and feature propagation

      ● Data dependent adjacency

      CUT[38],

      FastCUT[38],

      FRAGAN_O[31]

      ● CUT/FastCUT generator: ResNet

      ● FRAGAN_O: CUT structure, improved UNet++ generator

      ● PatchGAN discriminator

      		Unsupervised

      ● Proposes contrastive loss

      ● Trains on unpaired data

      ● Mode collapse may occur

      ● Performs poorly in complex scenes

      IRC[39]

      CCLGAN[40]

      ● Generator: UNet++

      ● Based on CUT architecture

      		Unsupervised

      ● Trains on unpaired data

      ● Improves generator

      ● Adds perceptual loss based on CUT

      		● Perceptual loss may degrade quality

      QS-Attn[41]

      CFSA-ICGAN[42]

      ● Generator: ResNet

      ● Based on CUT architecture

      		Unsupervised

      ● Trains on unpaired data

      ● Adds attention to contrastive loss to improve keypoint detection

      ● Contrastive loss is costly

      ● Simple generators underperform in complex scenarios

      DCLGAN[43]

      DC-Net[44]

      ● Generator: ResNet

      ● Improved based on CycleGAN architecture

      		Unsupervised

      ● Trains on unpaired data

      ● Proposes bilateral contrastive loss to improve CycleGAN’s cycle consistency loss

      ● GAN design is computationally expensive

      ● Simplistic generators struggle with complexity

      ● Color distortion

    • Table 1. Summary of infrared image colorization methods based on deep learning

      View table

      Table 1. Summary of infrared image colorization methods based on deep learning

      TypeMethodNetwork

      Learning

      strategy

      		StrengthWeakness
      CNNTIR[24]● UNetSupervised

      ● The first CNN infrared image colorization network

      ● Simple architecture

      ● Easy to implement

      ● Limited performance

      ● Requires data matching

      SNet[25]● Improved UNet-basedSupervised

      ● Auxiliary network designed within UNet

      ● Simple architecture

      ● Easy to implement

      ● Limited performance

      ● Requires precise data pairing for training

      AED[26]● Improved UNet-basedSupervised

      ● Use a weight-graph-based multiresolution fusion approach

      ● Simple architecture

      ● Easy to implement

      ● Tailored for near infrared (NIR)

      ● Needs data pairing

      GAN

      Pix2pix[27],

      TICC-GAN[28]

      ● Generator: UNet

      ● PatchGAN discriminator

      		Supervised

      ● Simple architecture

      ● Easy to implement

      ● Performs well in simple scenarios

      ● TICC-GAN integrates perceptual loss into Pix2pix

      ● Limited performance in complex scenes

      ● Needs data pairing

      DDGAN[29], LKAT-GAN[30], FRAGAN_P[31], MUGAN[32]

      ● Improved based on TICC-GAN

      ● DDGAN generator: dense connections

      ● LKAT-GAN generator: ViT+UNet

      ● FRAGAN_P generator: improved UNet++

      ● MUGAN: improved UNet3+

      		Supervised● The improved generator captures complex textures

      ● Needs exact data pairing

      ● High computation

      CycleGAN[33],

      FRAGAN_T[31]

      ● CycleGAN generator: ResNet

      ● FRAGAN_T: CycleGAN using improved UNet++

      ● PatchGAN discriminator

      		Unsupervised

      ● Training without paired data

      ● Introduces cycle consistency loss

      ● Mitigates mode collapse

      ● Colors appear unnatural

      ● Dual generators and discriminators increase cost

      PearlGAN[34],

      MornGAN[35],

      FoalGAN[36]

      		● Uses CycleGAN structure with an improved ResNet generatorUnsupervised

      ● PearlGAN introduces structured gradient alignment loss

      ● MornGAN incorporates semantic segmentation loss

      ● FoalGAN introduces subclass appearance consistency loss

      ● Effectively reduces subclass target loss

      ● Colors appear unnatural

      ● Segmentation loss depends on model quality

      ● Distortion persists in complex scenes

    • Table 2. Colorization test results of different methods

      View table

      Table 2. Colorization test results of different methods

      DatasetMethodPSNR（↑）SSIM（↑）MSE（↑）Colorfulness（↑）FID（↓）
      KAISTTIR13.060.420.0640.038167.928
      Pix2pix16.160.540.040.136129.833
      CUT13.600.430.0580.158152.386
      QS-Attn16.830.580.0330.41784.191
      CycleGAN14.730.440.0590.271224.823
      DCLGAN15.160.540.0550.060171.339

      FLIR

      		TIR12.130.470.0990.377253.398
      Pix2pix16.1380.5260.0480.198179.159
      CUT13.520.470.0910.052169.721
      QS-Attn16.070.540.0510.095176.752
      CycleGAN15.750.510.0560.207176.779
      DCLGAN15.150.540.0550.075172.146

      NIR

      		TIR15.520.510.0580.266193.416
      Pix2pix18.600.680.0320.319119.104
      CUT17.890.620.0340.090114.065
      QS-Attn18.830.690.0310.147105.650
      CycleGAN17.790.590.0360.06389.577
      DCLGAN17.810.620.0410.118239.018
    Tools

    Get Citation

    Copy Citation Text

    Xiubao Sui, Yuan Liu, Tong Jiang, Tingting Liu, Qian Chen. Review of Infrared Image Colorization Technology (Invited)[J]. Acta Optica Sinica, 2025, 45(17): 1720017

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Set citation alerts for article
    Save article for my favorites
    Paper Information

    Category: Optics in Computing

    Received: Jun. 8, 2025

    Accepted: Aug. 5, 2025

    Published Online: Sep. 3, 2025

    The Author Email: Xiubao Sui (sxb@njust.edu.cn)

    DOI:10.3788/AOS251235

    CSTR:32393.14.AOS251235

    Topics

    laser devices and laser physics
    Lasers and Laser Optics
    Laser physics
    laser manufacturing
    Instrumentation, Measurement and Metrology