Journals Articles Conferences News About CLP
Submit a paper Email Alert
Search
Search
Articles
Journals
News
Advanced Search
Top Searches
2D MaterialsTransformation opticsQuantum PhotonicsSmall lasersSemiconductor UV PhotonicsSupersymmetric microring laser arrays

Acta Optica Sinica, Volume. 40, Issue 2, 0210001(2020)

Infrared and Visible Image Fusion Method Based on Tikhonov Regularization and Detail Reconstruction

Xin Lu, Lin Yang, Min Li, and Xuewu Zhang*
Author Affiliations
  • College of Internet of Things Engineering, Hohai University, Changzhou, Jiangsu 213022, China
    • show less
    AbstractGet PDF(in Chinese)
    Figures&Tables (16)
    Equations (0)
    References (25)
    Cited By (1)
    Tools
    Paper Information
    Topics
    Figures & Tables(16)
    Algorithm flow diagram

    Fig. 1. Algorithm flow diagram

    Download full size
    Framework of image reconstruction model

    Fig. 2. Framework of image reconstruction model

    Download full size
    Generative network structure

    Fig. 3. Generative network structure

    Download full size
    Discriminant network structure

    Fig. 4. Discriminant network structure

    Download full size
    Network training process

    Fig. 5.

    Network training process

    Download full size
    Comparison of decomposition effects of different decomposition algorithms for “Smoke” scene. (a) Original image; (b) bilateral filtering; (c) guided filtering; (d) Gaussian pyramid; (e) wavelet transform; (f) Tikhonov regularization, α=2; (g) Tikhonov regularization, α=4 ; (h) Tikhonov regularization, α=8

    Fig. 6. Comparison of decomposition effects of different decomposition algorithms for “Smoke” scene. (a) Original image; (b) bilateral filtering; (c) guided filtering; (d) Gaussian pyramid; (e) wavelet transform; (f) Tikhonov regularization, α=2; (g) Tikhonov regularization, α=4 ; (h) Tikhonov regularization, α=8

    Download full size
    Comparison of decomposition effects of different decomposition algorithms for “Heather” scene. (a) Original image; (b) bilateral filtering; (c) guided filtering; (d) Gaussian pyramid; (e) wavelet transform; (f) Tikhonov regularization, α=2; (g) Tikhonov regularization, α=4 ; (h) Tikhonov regularization, α=8

    Fig. 7. Comparison of decomposition effects of different decomposition algorithms for “Heather” scene. (a) Original image; (b) bilateral filtering; (c) guided filtering; (d) Gaussian pyramid; (e) wavelet transform; (f) Tikhonov regularization, α=2; (g) Tikhonov regularization, α=4 ; (h) Tikhonov regularization, α=8

    Download full size
    Fusion effects of different algorithms in “Quad” scene. (a) Visible image; (b) infrared image; (c) DenseNet; (d) LatLRR; (e) VGG; (f) ResNet; (g) VSM; (h) QD; (i) GAN; (j) proposed algorithm

    Fig. 8. Fusion effects of different algorithms in “Quad” scene. (a) Visible image; (b) infrared image; (c) DenseNet; (d) LatLRR; (e) VGG; (f) ResNet; (g) VSM; (h) QD; (i) GAN; (j) proposed algorithm

    Download full size
    Fusion effects of different algorithms in “Smoke” scene. (a)Visible image; (b) infrared image; (c) DenseNet; (d) LatLRR; (e) VGG;(f) ResNet; (g) VSM; (h) QD; (i) GAN; (j) proposed algorithm

    Fig. 9. Fusion effects of different algorithms in “Smoke” scene. (a)Visible image; (b) infrared image; (c) DenseNet; (d) LatLRR; (e) VGG;(f) ResNet; (g) VSM; (h) QD; (i) GAN; (j) proposed algorithm

    Download full size
    Fusion effects of different algorithms in “Nato_camp” scene. (a) Visible image; (b) infrared image; (c) DenseNet; (d) LatLRR; (e) VGG; (f) ResNet; (g) VSM; (h) QD; (i) GAN; (j) proposed algorithm

    Fig. 10. Fusion effects of different algorithms in “Nato_camp” scene. (a) Visible image; (b) infrared image; (c) DenseNet; (d) LatLRR; (e) VGG; (f) ResNet; (g) VSM; (h) QD; (i) GAN; (j) proposed algorithm

    Download full size
    Fusion effects of different algorithms in blurred “Kaptein_1123” scene. (a) Visible image; (b) infrared image; (c) DenseNet; (d) LatLRR; (e) VGG; (f) ResNet; (g) VSM; (h) QD; (i) GAN; (j) proposed algorithm

    Fig. 11. Fusion effects of different algorithms in blurred “Kaptein_1123” scene. (a) Visible image; (b) infrared image; (c) DenseNet; (d) LatLRR; (e) VGG; (f) ResNet; (g) VSM; (h) QD; (i) GAN; (j) proposed algorithm

    Download full size
    Fusion effects of different algorithms in blurred “Heather” scene. (a) Visible image; (b) infrared image; (c) DenseNet; (d) LatLRR; (e) VGG; (f) ResNet; (g) VSM; (h) QD; (i) GAN; (j) proposed algorithm

    Fig. 12. Fusion effects of different algorithms in blurred “Heather” scene. (a) Visible image; (b) infrared image; (c) DenseNet; (d) LatLRR; (e) VGG; (f) ResNet; (g) VSM; (h) QD; (i) GAN; (j) proposed algorithm

    Download full size
    Fusion results of proposed algorithm in other scenes. (a) Steamer; (b) Bunker; (c) Street; (d) Jeep; (e) Soldier

    Fig. 13. Fusion results of proposed algorithm in other scenes. (a) Steamer; (b) Bunker; (c) Street; (d) Jeep; (e) Soldier

    Download full size
    Comparison of program running time of different algorithms

    Fig. 14. Comparison of program running time of different algorithms

    Download full size

    • Table 1. Parameter information of fully convolutional block

      View table

      Table 1. Parameter information of fully convolutional block

      NameKernel sizeStridePaddingOutput paddingOutput sizeBN
      Input----320×320×64-
      Conv1128×5×522-160×160×128√
      Conv2256×3×321-80×80×256√
      Conv3512×3×321-40×40×512√
      Conv4512×3×321-20×20×512√
      Conv51024×1×12--10×10×1024√
      DeConv1512×1×12-120×20×512√
      Add(Conv4+DeConv1)----20×20×512-
      DeConv2512×1×121140×40×512√
      Add(Conv3+DeConv2)----40×40×512-
      DeConv3256×3×321180×80×256√
      Add(Conv2+DeConv3)----80×80×256-
      DeConv4128×3×3211160×160×128√
      Add(Conv1+DeConv4)----160×160×128-
      DeConv564×5×5221320×320×64√
      Add(Input+DeConv5)----320×320×64-
      Output1×5×512-320×320×1-

    • Table 2. Objective evaluation results of different fusion methods

      View table

      Table 2. Objective evaluation results of different fusion methods

      ImageMetricDenseNetLatLRRVGGResNetVSMQDGANProposed method
      BunkerEN6.98076.81436.72776.80487.10737.07206.70437.1513
      SD31.40328.37926.13728.18636.00639.47125.94737.497
      SSIM1.27901.18061.15041.18501.21371.02721.14791.1767
      CC0.62700.63160.63450.63970.62230.53940.63260.6274
      SF0.01800.01980.02020.02030.02130.02090.02120.0213
      HeatherEN6.94136.60366.86436.73727.12346.79956.74117.0281
      SD32.67426.33730.48928.53738.26731.18530.19137.529
      SSIM1.01850.95731.01361.00441.04340.82660.90950.9770
      CC0.55750.55740.56490.56800.53800.46240.51060.5545
      SF0.01740.01860.01970.01940.02100.01910.02040.0212
      SandpathEN6.76426.25256.58666.54196.54196.73486.11596.7899
      SD29.81222.35726.15927.95227.95232.38518.07228.921
      SSIM0.90220.81950.88850.88110.88110.80180.77230.8921
      CC0.47800.47330.48580.46810.46810.42860.48160.4725
      SF0.02680.02620.02690.02730.02730.02700.02690.0273
      JeepEN7.14966.54707.13316.99497.02407.23586.79807.2032
      SD35.98823.43035.08033.68335.42739.73828.66938.340
      SSIM0.65120.50180.64600.62880.63220.58750.49700.6019
      CC0.36410.36170.36500.36710.35530.31740.28180.3428
      SF0.01320.01290.01510.01510.01610.01580.01520.0170
    Tools

    Get Citation

    Copy Citation Text

    Xin Lu, Lin Yang, Min Li, Xuewu Zhang. Infrared and Visible Image Fusion Method Based on Tikhonov Regularization and Detail Reconstruction[J]. Acta Optica Sinica, 2020, 40(2): 0210001

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Set citation alerts for article
    Save article for my favorites
    Paper Information

    Category: Image Processing

    Received: Jun. 24, 2019

    Accepted: Sep. 9, 2019

    Published Online: Jan. 2, 2020

    The Author Email: Zhang Xuewu (lab_112@126.com)

    DOI:10.3788/AOS202040.0210001

    Topics

    laser devices and laser physics
    Lasers and Laser Optics
    Laser physics
    laser manufacturing
    Instrumentation, Measurement and Metrology