Infrared Technology, Volume. 47, Issue 1, 52(2025)

Fusion of Hyperspectral and Multispectral Images Using a CNN Joint Multi-Scale Transformer

Guangxian XU, Weijie ZHOU, and Fei MA
Author Affiliations
  • School of Electronic and Information Engineering, Liaoning Technical University, Huludao 125105, China
  • show less
    References(29)

    [1] [1] YANG Q, XU Y, WU Z, et al. Hyperspectral And Multispectral Image Fusion Based On Deep Attention Network[C]//Proceedings of the 2019 10th Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing (WHISPERS), 2019: 1-5.

    [3] [3] Akbari H, Kosugi Y, Kojima K, et al. Detection and analysis of the intestinal ischemia using visible and invisible hyperspectral imaging[J]. IEEE Transactions on Biomedical Engineering, 2010, 57(8): 2011-2017.

    [4] [4] Zhihong P, Healey G, Prasad M, et al. Face recognition in hyperspectral images[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 25(12): 1552-1560.

    [5] [5] DIAN R, LI S, SUN B, et al. Recent advances and new guidelines on hyperspectral and multispectral image fusion[J]. Information Fusion, 2021, 69: 40-51.

    [6] [6] Fasbender D, Radoux J, Bogaert P. Bayesian data fusion for adaptable image pansharpening[J]. IEEE Transactions on Geoscience and Remote Sensing, 2008, 46(6): 1847-1857.

    [7] [7] CHEN Z, PU H, WANG B, et al. Fusion of hyperspectral and multispectral images: a novel framework based on generalization of pan-sharpening methods[J]. IEEE Geoscience and Remote Sensing Letters, 2014, 11(8): 1418-1422.

    [8] [8] SELVA M, AIAZZI B, BUTERA F, et al. Hyper-sharpening of hyperspectral data: A first approach[C]//Proceedings of the 2014 6th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS), 2014: 24-27.

    [9] [9] Zurita-Milla R, Clevers J G P W, Schaepman M E. Unmixing-based landsat TM and MERIS FR data fusion[J]. IEEE Geoscience and Remote Sensing Letters, 2008, 5(3): 453-457.

    [10] [10] Yokoya N, Yairi T, Iwasaki A. Coupled nonnegative matrix factorization unmixing for hyperspectral and multispectral data fusion[J]. IEEE Transactions on Geoscience and Remote Sensing, 2012, 50(2): 528-537.

    [11] [11] Lanaras C, Baltsavias E, Schindler K. Hyperspectral super-resolution by coupled spectral unmixing[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), 2015: 3586-3594.

    [12] [12] WEI Q, Dobigeon N, Tourneret J Y. Fast fusion of multi-band images based on solving a sylvester equation[J]. IEEE Transactions on Image Processing, 2015, 24(11): 4109-4121.

    [13] [13] Akhtar N, Shafait F, Mian A. Bayesian sparse representation for hyperspectral image super resolution[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015: 7-12.

    [15] [15] DIAN R, FANG L, LI S. Hyperspectral image super-resolution via non-local sparse tensor factorization[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017: 21-26.

    [16] [16] DIAN R, LI S, FANG L, et al. Nonlocal sparse tensor factorization for semiblind hyperspectral and multispectral image fusion[J]. IEEE Transactions on Cybernetics, 2020, 50(10): 4469-4480.

    [17] [17] LI S, DIAN R, FANG L, et al. Fusing hyperspectral and multispectral images via coupled sparse tensor factorization[J]. IEEE Transactions on Image Processing, 2018, 27(8): 4118-4130.

    [18] [18] Kanatsoulis C I, Fu X, Sidiropoulos N D, et al. Hyperspectral super-resolution: a coupled tensor factorization approach[J]. IEEE Transactions on Signal Processing, 2018, 66(24): 6503-6517.

    [19] [19] LI J, ZHENG K, YAO J, et al. Deep unsupervised blind hyperspectral and multispectral data fusion[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 1-5.

    [20] [20] WANG X, WANG X, ZHAO K, et al. FSL-Unet: full-scale linked unet with spatial–spectral joint perceptual attention for hyperspectral and multispectral image fusion[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1-14.

    [21] [21] DONG M, LI W, LIANG X, et al. MDCNN: multispectral pansharpening based on a multiscale dilated convolutional neural network[J]. Journal of Applied Remote Sensing, 2021, 15: 036516.

    [22] [22] Benzenati T, Kessentini Y, Kallel A. Pansharpening approach via two-stream detail injection based on relativistic generative adversarial networks[J]. Expert Systems with Applications, 2022, 188: 115996.

    [23] [23] FU Y, XU T, WU X, et al. PPT Fusion: pyramid patch transformerfor a case study in image fusion[J]. arXiv preprint arXiv: 2107.13967, 2021.

    [24] [24] GAO S H, CHENG M M, ZHAO K, et al. Res2Net: a new multi-scale backbone architecture[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(2): 652-662.

    [25] [25] LIU Z, LIN Y, CAO Y, et al. Swin Transformer: hierarchical vision transformer using shifted windows[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021: 10-17.

    [26] [26] Kingma D P, BA J. Adam: a method for stochastic optimization[J/OL]. arXiv: 1412.6980, 2014, https://doi.org/10.48550/arXiv.1412.6980.

    [27] [27] ZHANG X, HUANG W, WANG Q, et al. SSR-NET: spatial–spectral reconstruction network for hyperspectral and multispectral image fusion[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(7): 5953-5965.

    [28] [28] LIU X, LIU Q, WANG Y. Remote sensing image fusion based on two-stream fusion network[J]. Information Fusion, 2020, 55: 1-15.

    [29] [29] XIE Q, ZHOU M, ZHAO Q, et al. MHF-Net: an interpretable deep network for multispectral and hyperspectral image fusion[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(3): 1457-1473.

    [30] [30] HU J F, HUANG T Z, DENG L J, et al. Hyperspectral image super-resolution via deep spatiospectral attention convolutional neural networks[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(12): 7251-7265.

    [31] [31] HU J F, HUANG T Z, DENG L J, et al. Fusformer: a transformer-based fusion network for hyperspectral image super-resolution[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 1-5.

    Tools

    Get Citation

    Copy Citation Text

    XU Guangxian, ZHOU Weijie, MA Fei. Fusion of Hyperspectral and Multispectral Images Using a CNN Joint Multi-Scale Transformer[J]. Infrared Technology, 2025, 47(1): 52

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category:

    Received: Jul. 21, 2023

    Accepted: Feb. 18, 2025

    Published Online: Feb. 18, 2025

    The Author Email:

    DOI:

    CSTR:32186.14.

    Topics