Journals Articles Conferences News About CLP
Submit a paper Email Alert
Search
Search
Articles
Journals
News
Advanced Search
Top Searches
2D MaterialsTransformation opticsQuantum PhotonicsSmall lasersSemiconductor UV PhotonicsSupersymmetric microring laser arrays

Laser & Optoelectronics Progress, Volume. 60, Issue 16, 1610013(2023)

Infrared and Visible Image Fusion with Convolutional Neural Network and Transformer

Yang Yang, Zhennan Ren*, and Beichen Li
Author Affiliations
  • School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China
    • show less
    AbstractGet PDF(in Chinese)
    Figures&Tables (10)
    Equations (18)
    References (42)
    Cited By (1)
    Tools
    Paper Information
    Topics
    Figures & Tables(10)
    Overall structure of the proposed model

    Fig. 1. Overall structure of the proposed model

    Download full size
    Concrete structure of the Encoder

    Fig. 2. Concrete structure of the Encoder

    Download full size
    Fusion strategy

    Fig. 3. Fusion strategy

    Download full size
    Fusion results of the “Street” image. (a) Infrared image; (b) visible image; (c) RP; (d) Wavelet; (e) ResNet-ZCA; (f) DenseFuse; (g) Dual-Branch; (h) FusionGAN; (i) GANMcC; (j) proposed method

    Fig. 4. Fusion results of the “Street” image. (a) Infrared image; (b) visible image; (c) RP; (d) Wavelet; (e) ResNet-ZCA; (f) DenseFuse; (g) Dual-Branch; (h) FusionGAN; (i) GANMcC; (j) proposed method

    Download full size
    Fusion results of the “Kaptein_1123” image. (a) Infrared image; (b) visible image; (c) RP; (d) Wavelet; (e) ResNet-ZCA; (f) DenseFuse; (g) Dual-Branch; (h) FusionGAN; (i) GANMcC; (j) proposed method

    Fig. 5. Fusion results of the “Kaptein_1123” image. (a) Infrared image; (b) visible image; (c) RP; (d) Wavelet; (e) ResNet-ZCA; (f) DenseFuse; (g) Dual-Branch; (h) FusionGAN; (i) GANMcC; (j) proposed method

    Download full size
    Fusion results of the “Kaptein_1654” image. (a) Infrared image; (b) visible image; (c) RP; (d) Wavelet; (e) ResNet-ZCA; (f) DenseFuse; (g) Dual-Branch; (h) FusionGAN; (i) GANMcC; (j) proposed method

    Fig. 6. Fusion results of the “Kaptein_1654” image. (a) Infrared image; (b) visible image; (c) RP; (d) Wavelet; (e) ResNet-ZCA; (f) DenseFuse; (g) Dual-Branch; (h) FusionGAN; (i) GANMcC; (j) proposed method

    Download full size
    Fusion results of the “Bunker” image. (a) Infrared image; (b) visible image; (c) RP; (d) Wavelet; (e) ResNet-ZCA; (f) DenseFuse; (g) Dual-Branch; (h) FusionGAN; (i) GANMcC; (j) proposed method

    Fig. 7. Fusion results of the “Bunker” image. (a) Infrared image; (b) visible image; (c) RP; (d) Wavelet; (e) ResNet-ZCA; (f) DenseFuse; (g) Dual-Branch; (h) FusionGAN; (i) GANMcC; (j) proposed method

    Download full size
    Six objective metrics of different fusion models on TNO dataset. (a) En; (b) SD; (c) SF; (d) MI; (e) SCD; (f) Q_abf

    Fig. 8. Six objective metrics of different fusion models on TNO dataset. (a) En; (b) SD; (c) SF; (d) MI; (e) SCD; (f) Q_abf

    Download full size
    Subjective results of the ablation experiment

    Fig. 9. Subjective results of the ablation experiment

    Download full size

    • Table 1. Average value of objective results of the ablation experiment

      View table

      Table 1. Average value of objective results of the ablation experiment

      ModelEnSDSFMISCDQ_abf
      cnn&mean6.554628.20876.02952.23801.71050.3059
      cnn+trans&mean6.657431.77636.52092.26481.72490.3110
      cnn&spa_fus6.777535.12027.46653.29541.68790.4115
      Proposed method6.963940.72717.95352.69491.76450.3695
    Tools

    Get Citation

    Copy Citation Text

    Yang Yang, Zhennan Ren, Beichen Li. Infrared and Visible Image Fusion with Convolutional Neural Network and Transformer[J]. Laser & Optoelectronics Progress, 2023, 60(16): 1610013

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Set citation alerts for article
    Save article for my favorites
    Paper Information

    Category: Image Processing

    Received: Aug. 12, 2022

    Accepted: Oct. 27, 2022

    Published Online: Aug. 18, 2023

    The Author Email: Ren Zhennan (Ren2151311@163.com)

    DOI:10.3788/LOP222265

    Topics

    laser devices and laser physics
    Lasers and Laser Optics
    Laser physics
    laser manufacturing
    Instrumentation, Measurement and Metrology