Optics and Precision Engineering, Volume. 31, Issue 5, 644(2023)
Semantic segmentation of multi-source remote sensing data self-adaptive fusion with independent branch network
Existing deep learning-based terrain classification methods are mainly for remote sensing imagery; however, the spatial information of point clouds is underutilized. Specifically, the fusion of heterologous features is insufficient for point clouds and imagery. To utilize multi-source features fully, we propose a self-adaptive fusion classification method of multi-source remote sensing data based on independent branch network in this study. First, three-dimensional (3D) and two-dimensional (2D) networks are used to extract the semantic features of registered LiDAR point clouds and remote sensing imagery. From the 3D space, the features of imagery are then sampled and aligned with those of point clouds. Finally, a nonlinear self-adaptive feature fusion module is proposed to realize the fusion of multi-source semantic features. The experimental results indicate that the proposed method achieves an average classification accuracy of 85.87% on the vegetation, building, and ground of the ISPRS multi-source remote sensing dataset. Through network training, multi-source remote sensing data can be more data feature-adaptive fused and classified; further, the accuracy is significantly improved by 10.12% compared with the 3D classification result. The proposed independent branch fusion network can realize interactive learning and deep fusion of 2D and 3D data, and it provide a new idea for terrain classification based on remote sensing multimodal data fusion.
Get Citation
Copy Citation Text
Mofan DAI, Qing XU, Shuai XING, Pengcheng LI. Semantic segmentation of multi-source remote sensing data self-adaptive fusion with independent branch network[J]. Optics and Precision Engineering, 2023, 31(5): 644
Category: Three-dimensional topographic mapping
Received: Aug. 8, 2022
Accepted: --
Published Online: Apr. 4, 2023
The Author Email: XING Shuai (xing972403@163.com)