Journal of Terahertz Science and Electronic Information Technology , Volume. 21, Issue 7, 939(2023)

Spatiotemporal fusion of remote sensing images based on multi-level feature compensation

LIU Wenjie1,2, LI Yujia1,2, BAI Menghao1,2, ZHANG Liping1,2, and LEI Dajiang1,2、*
Author Affiliations
  • 1[in Chinese]
  • 2[in Chinese]
  • show less
    References(37)

    [1] [1] TAN Zhenyu, DI Liping, ZHANG Mingda, et al. An enhanced deep convolutional model for spatiotemporal image fusion[J]. Remote Sensing, 2019,11(24):2898. doi:10.3390/rs11242898.

    [2] [2] PASTOR-GUZMAN J,BROWN L,MORRIS H,et al. The sentinel-3 OLCI Terrestrial Chlorophyll Index(OTCI):algorithm improvements, spatiotemporal consistency and continuity with the MERIS archive[J]. Remote Sensing, 2020,12(16):2652. doi: 10.3390/rs12162652.

    [3] [3] GUHA S,GOVIL H. An assessment on the relationship between land surface temperature and normalized difference vegetation index[J]. Environment,Development and Sustainability, 2021,23(2):1944-1963. doi:10.1007/s10668-020-00657-6.

    [4] [4] SHEN Huanfeng, MENG Xiangchao, ZHANG Liangpei. An integrated framework for the spatio-temporal-spectral fusion of remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016,54(12):7135-7148. doi:10.1109/TGRS. 2016.2596290.

    [5] [5] CHEN Bin, HUANG Bo, XU Bin. Comparison of spatiotemporal fusion models: a review[J]. Remote Sensing, 2015, 7(2): 1798-1835. doi:10.3390/rs70201798.

    [6] [6] GAO F,MASEK J,SCHWALLER M,et al. On the blending of the Landsat and MODIS surface reflectance:predicting daily landsat surface reflectance[J]. IEEE Transactions on Geoscience and Remote Sensing, 2006, 44(8): 2207-2218. doi: 10.1109/TGRS. 2006.872081.

    [7] [7] HILKER T,WULDER M A,COOPS N C,et al. Generation of dense time series synthetic Landsat data through data blending with MODIS using a spatial and temporal adaptive reflectance fusion model[J]. Remote Sensing of Environment, 2009,113(9):1988-1999. doi:10.1016/j.rse.2009.05.011.

    [8] [8] KHALEGHI B,KHAMIS A,KARRAY F O,et al. Multisensor data fusion:a review of the state-of-the-art[J]. Information Fusion, 2013,14(1):28-44. doi:10.1016/j.inffus.2011.08.001.

    [9] [9] BELGIU M, STEIN A. Spatiotemporal image fusion in remote sensing[J]. Remote Sensing, 2019, 11(7): 818. doi: 10.3390/ rs11070818.

    [10] [10] ZHANG Hongyan, SONG Yiyao, HAN Chang, et al. Remote sensing image spatiotemporal fusion using a generative adversarial network[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020,59(5):4273-4286. doi:10.1109/TGRS.2020.3010530.

    [11] [11] MASEK J G,WULDER M A,MARKHAM B,et al. Landsat 9:empowering open science and applications through continuity[J]. Remote Sensing of Environment, 2020(248):111968. doi:10.1016/j.rse.2020.111968.

    [12] [12] TAN Zhenyu, YUE Peng, DI Liping, et al. Deriving high spatiotemporal remote sensing images using deep convolutional network[J]. Remote Sensing, 2018,10(7):1066. doi:10.3390/rs10071066.

    [14] [14] ACERBI-JUNIOR F W,CLEVERS J,SCHAEPMAN M E. The assessment of multi-sensor image fusion using wavelet transforms for mapping the Brazilian Savanna[J]. International Journal of Applied Earth Observation and Geoinformation, 2006,8(4):278-288. doi:10.1016/j.jag.2006.01.001.

    [15] [15] PENG Yidong,LI Weisheng,LUO Xiaobo,et al. Integrated fusion framework based on semicoupled sparse tensor factorization for spatio-temporal-spectral fusion of remote sensing images[J]. Information Fusion, 2021(65):21-36. doi:10.1016/j.inffus. 2020. 08.013.

    [18] [18] HE Lin, ZHU Jiawei, LI Jun, et al. Spectral-fidelity convolutional neural networks for hyperspectral pansharpening[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020(13): 5898-5914. doi: 10.1109/JSTARS. 2020.3025040.

    [20] [20] HUANG Bo, ZHANG Hankui, SONG Huihui, et al. Unified fusion of remote-sensing imagery: generating simultaneously high-resolution synthetic spatial-temporal-spectral earth observations[J]. Remote Sensing Letters, 2013,4(6):561-569. doi:10.1080/ 2150704X.2013.769283.

    [21] [21] XUE Jie,LEUNG Y,FUNG T. A Bayesian data fusion approach to spatio-temporal fusion of remotely sensed images[J]. Remote Sensing, 2017,9(12):1310. doi:10.3390/rs9121310.

    [22] [22] CHEN Junfu,PAN Yue,CHEN Yang. Remote sensing image fusion based on Bayesian GAN[J/OL]. arXiv:2009.09465, 2020.

    [23] [23] ZHUANG Qifeng, SHI Yintao, SHAO Hua, et al. Evaluating the SSEBop and RSPMPT models for irrigated fields daily evapotranspiration mapping with MODIS and CMADS data[J]. Agriculture, 2021,11(5):424. doi:10.3390/agriculture11050424.

    [24] [24] HUANG Bo, SONG Huihui. Spatiotemporal reflectance fusion via sparse representation[J]. IEEE Transactions on Geoscience and Remote Sensing, 2012,50(10):3707-3716. doi:10.1109/TGRS.2012.2186638.

    [25] [25] SONG Huihui, LIU Qingshan, WANG Guojie, et al. Spatiotemporal satellite image fusion using deep convolutional neural networks[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11(3): 821-829. doi: 10.1109/JSTARS.2018.2797894.

    [26] [26] SONG Huihui, HUANG Bo. Spatiotemporal satellite image fusion through one-pair image learning[J]. IEEE Transactions on Geoscience and Remote Sensing, 2012,51(4):1883-1896. doi:10.1109/TGRS.2012.2213095.

    [28] [28] LIU Xun, DENG Chenwei, CHANUSSOT J, et al. StfNet: a two-stream convolutional neural network for spatiotemporal image fusion[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019,57(9):6552-6564. doi:10.1109/TGRS.2019.2907310.

    [29] [29] ODENA A, DUMOULIN V, OLAH C. Deconvolution and checkerboard artifacts[J]. Distill, 2016, 1(10): e3. doi: 10.23915/ distill.00003.

    [30] [30] LUO Chunjie,ZHAN Jianfeng,WANG Lei,et al. Extended batch normalization[J/OL]. arXiv:2003.05569, 2020.

    [31] [31] ZHANG Yongjun, LIU Chi, SUN Mingwei, et al. Pan-sharpening using an efficient bidirectional pyramid network[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019,57(8):5549-5563. doi:10.1109/TGRS.2019.2900419.

    [32] [32] JOHNSON J, ALAHI A, FEI-FEI L. Perceptual losses for real-time style transfer and super-resolution[C]// European Conference on Computer Vision. [S.l.]:Springer,Cham, 2016:694-711. doi:10.1007/978-3-319-46475-6_43.

    [33] [33] LEI Dajiang,BAI Menghao,ZHANG Liping,et al. Convolution neural network with edge structure loss for spatiotemporal remote sensing image fusion[J]. International Journal of Remote Sensing, 2022,43(3):1015-1036. doi:10.1080/01431161.2022.2030070.

    [34] [34] CHEN Yinpeng,DAI Xiyang,LIU Mengchen,et al. Dynamic ReLu[C]// European Conference on Computer Vision. [S.l.]:Springer, Cham, 2020:351-367. doi:10.1007/978-3-030-58529-7_21.

    [35] [35] KOKHLIKYAN N,MIGLANI V,MARTIN M,et al. Captum:a unified and generic model interpretability library for pytorch[J/OL]. arXiv:2009.07896, 2020.

    [36] [36] LIU Meng, YANG Wei, ZHU Xiaolin, et al. An Improved Flexible Spatiotemporal Data Fusion(IFSDAF) method for producing high spatiotemporal resolution normalized difference vegetation index time series[J]. Remote Sensing of Environment, 2019 (227):74-89. doi:10.1016/j.rse.2019.03.012.

    [37] [37] WANG Qunming, TANG Yijie, TONG Xiaohua, et al. Virtual image pair-based spatio-temporal fusion[J]. Remote Sensing of Environment, 2020(249):112009. doi:10.1016/j.rse.2020.112009.

    [38] [38] AO Z,SUN Y,XIN Q C. Constructing 10 m NDVI time series from Landsat 8 and sentinel 2 images using convolutional neural networks[J]. IEEE Geoscience and Remote Sensing Letters, 2020,18(8):1461-1465. doi:10.1109/LGRS.2020.3003322.

    [39] [39] ALCARAS E,PARENTE C,VALLARIO A. Automation of pansharpening methods for pleiades images using GIS basic functions[J]. Remote Sensing, 2021,13(8):1550. doi:10.3390/rs13081550.

    [40] [40] LI Weisheng,XIANG Minghao,LIANG Xuesong. MDCwFB:a multilevel dense connection network with feedback connections for pansharpening[J]. Remote Sensing, 2021,13(11):2218. doi:10.3390/rs13112218.

    [41] [41] XIE Weiying,CUI Yuhang,LI Yunsong,et al. HPGAN:hyperspectral pansharpening using 3D generative adversarial networks[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020,59(1):463-477. doi:10.1109/TGRS.2020.2994238.

    [42] [42] SCARPA G,CIOTOLA M. Full-resolution quality assessment for pansharpening[J/OL]. arXiv:2108.06144, 2021. doi: 10.3390/ rs14081808.

    Tools

    Get Citation

    Copy Citation Text

    LIU Wenjie, LI Yujia, BAI Menghao, ZHANG Liping, LEI Dajiang. Spatiotemporal fusion of remote sensing images based on multi-level feature compensation[J]. Journal of Terahertz Science and Electronic Information Technology , 2023, 21(7): 939

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category:

    Received: Sep. 30, 2022

    Accepted: --

    Published Online: Jan. 17, 2024

    The Author Email: Dajiang LEI (leidj@cqupt.edu.cn)

    DOI:10.11805/tkyda2022191

    Topics