Acta Photonica Sinica, Volume. 52, Issue 4, 0428002(2023)

Remote Sensing Image Fusion Based on Two-branch U-shaped Transformer

Wensheng FAN, Fan LIU*, and Ming LI
Author Affiliations
  • College of Data Science, Taiyuan University of Technology, Jinzhong 030600, China
  • show less
    References(36)

    [1] GHAMISI P, RASTI B, YOKOYA N et al. Multisource and multitemporal data fusion in remote sensing: a comprehensive review of the state of the art[J]. IEEE Geoscience and Remote Sensing Magazine, 7, 6-39(2019).

    [2] BOVOLO F, BRUZZONE L, CAPOBIANCO L et al. Analysis of the effects of pansharpening in change detection on vhr images[J]. IEEE Geoscience and Remote Sensing Letters, 7, 53-57(2010).

    [3] GILBERTSON J, KEMP J, NIEKERK A. Effect of pan-sharpening multi-temporal landsat 8 imagery for crop type differentiation using different classification techniques[J]. Computers and Electronics in Agriculture, 134, 151-159(2017).

    [4] VIVONE G, MURA M, GARZELLI A et al. A new benchmark based on recent advances in multispectral pansharpening: revisiting pansharpening with classical and emerging pansharpening methods[J]. IEEE Geoscience and Remote Sensing Magazine, 9, 53-81(2021).

    [5] CARPER W, LILLESAND T, KIEFER R. The use of intensity-hue-saturation transformations for merging spot panchromatic and multi-spectral image data[J]. Photogrammetric Engineering and Remote Sensing, 56, 459-467(1990).

    [6] KWARTENG P, CHAVEZ A. Extracting spectral contrast in landsat thematic mapper image data using selective principal component analysis[J]. Photogrammetric Engineering and Remote Sensing, 55, 339-348(1989).

    [7] AIAZZI B, BARONTI S, SELVA M. Improving component substitution pansharpening through multivariate regression of ms+pan data[J]. IEEE Transactions on Geoscience and Remote Sensing, 45, 3230-3239(2007).

    [8] GARZELLI A, NENCINI F, CAPOBIANCO L. Optimal mmse pan sharpening of very high resolution multispectral images[J]. IEEE Transactions on Geoscience and Remote Sensing, 46, 228-236(2008).

    [9] ZHANG Jing, CHEN Hongtao, LIU Fan. Remote sensing image fusion based on multivariate empirical mode decomposition and weighted least squares filter[J]. Acta Photonica Sinica, 48, 0510003(2019).

    [10] KIM Y, LEE C, HAN D et al. Improved additive-wavelet image fusion[J]. IEEE Geoscience and Remote Sensing Letters, 8, 263-267(2011).

    [11] PRADHAN P S, KING R L, YOUNAN N H et al. Estimation of the number of decomposition levels for a wavelet-based multiresolution multisensor image fusion[J]. IEEE Transactions on Geoscience and Remote Sensing, 44, 3674-3686(2006).

    [12] AIAZZI B, ALPARONE L, BARONTI S et al. An mtf-based spectral distortion minimizing model for pan-sharpening of very high resolution multispectral images of urban areas[C], 90-94(2003).

    [13] LIU Fan. Remote sensing image fusion based on wavelet kernel filter and sparse representation[D](2014).

    [14] VICINANZA M R, RESTAINO R, VIVONE G et al. A pansharpening method based on the sparse representation of injected details[J]. IEEE Geoscience and Remote Sensing Letters, 12, 180-184(2015).

    [15] PEI Xiaopeng. Remote sensing image fusion based on sparse representation[D](2018).

    [16] PALSSON F, SVEINSSON J R, ULFARSSON M O. A new pansharpening algorithm based on total variation[J]. IEEE Geoscience and Remote Sensing Letters, 11, 318-322(2014).

    [17] DONG C, LOY C C, HE K et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38, 295-307(2016).

    [18] MASI G, COZZOLINO D, VERDOLIVA L et al. Pansharpening by convolutional neural networks[J]. Remote Sensing, 8, 594(2016).

    [19] YUAN Q, WEI Y, MENG X et al. A multiscale and multidepth convolutional neural network for remote sensing imagery pan-sharpening[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11, 978-989(2018).

    [20] YANG J, FU X, HU Y et al. Pannet: a deep network architecture for pan-sharpening[C], 5449-5457(2017).

    [21] SHAO Z, CAI J. Remote sensing image fusion with deep convolutional neural network[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11, 1656-1669(2018).

    [22] LIU Q, ZHOU H, XU Q et al. Psgan: a generative adversarial network for remote sensing image pan-sharpening[J]. IEEE Transactions on Geoscience and Remote Sensing, 59, 10227-10242(2021).

    [23] OZCELIK F, ALGANCI U, SERTEL E et al. Rethinking CNN-based pansharpening: guided colorization of panchromatic images via GANs[J]. IEEE Transactions on Geoscience and Remote Sensing, 59, 3486-3501(2021).

    [24] RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C], 234-241(2015).

    [25] SHEN H, JIANG M, LI J et al. Spatial-spectral fusion by combining deep learning and variational model[J]. IEEE Transactions on Geoscience and Remote Sensing, 57, 6169-6181(2019).

    [26] DENG L, VIVONE G, JIN C et al. Detail injection-based deep convolutional neural networks for pansharpening[J]. IEEE Transactions on Geoscience and Remote Sensing, 59, 6995-7010(2021).

    [27] VASWANI A, SHAZEER N, PARMAR N et al. Attention is all you need[C], 5998-6008(2017).

    [28] LIU Z, LIN Y, CAO Y et al. Swin transformer: hierarchical vision transformer using shifted windows[C], 10012-10022(2021).

    [29] WALD L, RANCHIN T, MANGOLINI M. Fusion of satellite images of different spatial resolutions: assessing the quality of resulting images[J]. Photogrammetric Engineering and Remote Sensing, 63, 691-699(1997).

    [30] YUHAS R, GOETZ A, BOARDMAN J. Discrimination among semi-arid landscape endmembers using the spectral angle mapper (SAM) algorithm[C], 147-149(1992).

    [31] WALD L[M]. Data fusion: definitions and architectures: fusion of images of different spatial resolutions(2002).

    [32] ZHOU J, CIVCO D, SILANDER J. A wavelet transform method to merge landsat tm and spot panchromatic data[J]. International Journal of Remote Sensing, 19, 743-757(1998).

    [33] ALPARONE L, BARONTI S, GARZELLI A et al. A global quality measurement of pan-sharpened multispectral imagery[J]. IEEE Geoscience and Remote Sensing Letters, 1, 313-317(2004).

    [34] GARZELLI A, NENCINI F. Hypercomplex quality assessment of multi/hyperspectral images[J]. IEEE Geoscience and Remote Sensing Letters, 6, 662-665(2009).

    [35] AIAZZI B, ALPARONE L, BARONTI S et al. Full-scale assessment of pansharpening methods and data products[C], 9244, 1-12(2014).

    [36] LEI D, CHEN H, ZHANG L et al. Nlrnet: an efficient nonlocal attention resnet for pansharpening[J]. IEEE Transactions on Geoscience and Remote Sensing, 60, 1-13(2021).

    Tools

    Get Citation

    Copy Citation Text

    Wensheng FAN, Fan LIU, Ming LI. Remote Sensing Image Fusion Based on Two-branch U-shaped Transformer[J]. Acta Photonica Sinica, 2023, 52(4): 0428002

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Remote Sensing and Sensors

    Received: Oct. 19, 2022

    Accepted: Dec. 26, 2022

    Published Online: Jun. 21, 2023

    The Author Email: Fan LIU (liufan@tyut.edu.cn)

    DOI:10.3788/gzxb20235204.0428002

    Topics