Optics and Precision Engineering, Volume. 31, Issue 5, 644(2023)

Semantic segmentation of multi-source remote sensing data self-adaptive fusion with independent branch network

Mofan DAI1...2, Qing XU1,2, Shuai XING1,* and Pengcheng LI1 |Show fewer author(s)
Author Affiliations
  • 1Institute of Geospatial Information, Strategic Support Force Information Engineering University, Zhengzhou45000, China
  • 2Collaborative Innovation Center of Geo-information Technology for Smart Central Plains, Zhengzhou450001, China
  • show less

    Existing deep learning-based terrain classification methods are mainly for remote sensing imagery; however, the spatial information of point clouds is underutilized. Specifically, the fusion of heterologous features is insufficient for point clouds and imagery. To utilize multi-source features fully, we propose a self-adaptive fusion classification method of multi-source remote sensing data based on independent branch network in this study. First, three-dimensional (3D) and two-dimensional (2D) networks are used to extract the semantic features of registered LiDAR point clouds and remote sensing imagery. From the 3D space, the features of imagery are then sampled and aligned with those of point clouds. Finally, a nonlinear self-adaptive feature fusion module is proposed to realize the fusion of multi-source semantic features. The experimental results indicate that the proposed method achieves an average classification accuracy of 85.87% on the vegetation, building, and ground of the ISPRS multi-source remote sensing dataset. Through network training, multi-source remote sensing data can be more data feature-adaptive fused and classified; further, the accuracy is significantly improved by 10.12% compared with the 3D classification result. The proposed independent branch fusion network can realize interactive learning and deep fusion of 2D and 3D data, and it provide a new idea for terrain classification based on remote sensing multimodal data fusion.

    Tools

    Get Citation

    Copy Citation Text

    Mofan DAI, Qing XU, Shuai XING, Pengcheng LI. Semantic segmentation of multi-source remote sensing data self-adaptive fusion with independent branch network[J]. Optics and Precision Engineering, 2023, 31(5): 644

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Three-dimensional topographic mapping

    Received: Aug. 8, 2022

    Accepted: --

    Published Online: Apr. 4, 2023

    The Author Email: XING Shuai (xing972403@163.com)

    DOI:10.37188/OPE.20233105.0644

    Topics