Semantic Segmentation of Dual-Source Remote Sensing Images Based on Gated Attention and Multiscale Residual Fusion

Wen Guo; Hong Yang; Chang Liu

doi:10.3788/LOP240534

Laser & Optoelectronics Progress, Volume. 61, Issue 18, 1837015(2024)

Semantic Segmentation of Dual-Source Remote Sensing Images Based on Gated Attention and Multiscale Residual Fusion

Wen Guo¹, Hong Yang¹, and Chang Liu^2、*

Author Affiliations

¹School of science, Beijing Information Science and Technology University, Beijing 100029, China

²Institute of Applied Mathematics, Beijing Information Science and Technology University, Beijing 100101, China

show less

Abstract Get PDF(in Chinese)

Figures & Tables(12)

Fig. 1. Overall structure diagram of STAM-SegNet

Fig. 2. AGCA module

Fig. 3. AGSA module

Fig. 4. MFRF module

Fig. 5. Overall technical flowchart of the experiment

Download full size

Fig. 6. Corresponding prediction effect diagram of comparative experimental results of different models on the Vaihingen dataset

Download full size

Fig. 7. Corresponding prediction effect diagram of comparative experimental results of different models on the Potsdam dataset

Download full size

Table 1. Dataset preprocessing and training configuration
View table
Table 1. Dataset preprocessing and training configuration
Item ISPRS Vaihingen ISPRS Potsdam
Input size 512 $\times$ 512 512 $\times$ 512
Bands used IRRG，DSM IRRG，DSM
Batch size 4 4
Train number 16 17
Val number 4 5
Test number 17 14

Table 2. Comparison of evaluation results of ablation experiment on the Vaihingen test set
View table
Table 2. Comparison of evaluation results of ablation experiment on the Vaihingen test set
Baseline（Swin Transformer） AGCA AGSA MFRF Mean F1 /% OA /%
√ × × × 88.59 89.36
√ √ × × 89.08 89.94
√ √ √ × 89.45 90.47
√ √ √ √ 89.66 90.73

Table 3. Comparative experimental results of dual-source data feature fusion on the Vaihingen dataset
View table
Table 3. Comparative experimental results of dual-source data feature fusion on the Vaihingen dataset
Method Input Mean F1 /% OA /%
Baseline IRRG 88.02 88.94
Add IRRG+DSM 88.73 89.52
Concat IRRG+DSM 88.87 89.77
AGCA IRRG+DSM 89.66 90.73

Table 4. Comparison results of multiple evaluation metrics for each model on the Vaihingen testset

View table

Table 4. Comparison results of multiple evaluation metrics for each model on the Vaihingen testset

Method	Backbone	F1-score /%					Mean F1 /%	OA /%	GFLOPs /GB	Params /10⁶
Method	Backbone	Impervios_surface	Building	Low_vegetable	Tree	Car	Mean F1 /%	OA /%	GFLOPs /GB	Params /10⁶
DeepLabV3+	Resnet101	91.52	94.08	81.67	88.64	82.47	87.68	89.24	193.94	49.58
UperNet	Resnet101	91.48	93.89	81.21	88.34	86.14	88.21	89.07	243.97	70.60
DANet	Resnet101	92.15	94.80	83.19	89.17	86.54	89.17	90.01	276.73	66.45
TransUNet	Resnet101+VIT	92.08	94.91	83.02	88.98	86.57	89.11	89.95	803.4	90.7
Swin-UNet	Swin Transformer	92.49	95.64	83.12	89.31	86.37	89.38	90.44	349.72	82.89
STAM-SegNet	Swin Transformer	92.56	95.96	83.82	89.34	86.61	89.66	90.73	233.21	60.47

Table 5. Comparison results of multiple evaluation metrics for each model on the Potsdam testset

View table

Table 5. Comparison results of multiple evaluation metrics for each model on the Potsdam testset

Method	Backbone	F1-score /%					Mean F1 /%	OA /%	GFLOPS /GB	Params /10⁶
Method	Backbone	Impervios_surface	Building	Low_vegetable	Tree	Car	Mean F1 /%	OA /%	GFLOPS /GB	Params /10⁶
DeepLabV3+	Resnet101	92.08	95.65	86.16	87.13	95.65	91.33	89.67	193.94	49.58
UperNet	Resnet101	92.52	95.98	86.38	87.45	95.85	91.64	90.01	243.97	70.60
DANet	Resnet101	92.60	96.16	86.55	87.26	95.90	91.69	90.10	276.73	66.45
TransUNet	Resnet101+VIT	92.34	96.10	86.23	87.29	96.03	91.60	90.07	803.4	90.7
Swin-UNet	Swin Transformer	92.91	96.91	87.15	88.37	95.99	92.27	90.72	349.72	82.89
STAM-SegNet	Swin Transformer	93.58	97.13	87.51	89.19	96.32	92.75	91.23	233.21	60.47

Tools

Get Citation

Copy Citation Text

Wen Guo, Hong Yang, Chang Liu. Semantic Segmentation of Dual-Source Remote Sensing Images Based on Gated Attention and Multiscale Residual Fusion[J]. Laser & Optoelectronics Progress, 2024, 61(18): 1837015

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites

Paper Information

Category: Digital Image Processing

Received: Jan. 15, 2024

Accepted: Feb. 26, 2024

Published Online: Sep. 14, 2024

The Author Email: Chang Liu (liuchang@bistu.edu.cn)

DOI:10.3788/LOP240534

CSTR:32186.14.LOP240534

Topics

laser devices and laser physics

Lasers and Laser Optics

Laser physics

laser manufacturing

Instrumentation, Measurement and Metrology

Table 1. Dataset preprocessing and training configuration

Table 1. Dataset preprocessing and training configuration

Table 2. Comparison of evaluation results of ablation experiment on the Vaihingen test set

Table 2. Comparison of evaluation results of ablation experiment on the Vaihingen test set

Table 3. Comparative experimental results of dual-source data feature fusion on the Vaihingen dataset

Table 3. Comparative experimental results of dual-source data feature fusion on the Vaihingen dataset

Table 4. Comparison results of multiple evaluation metrics for each model on the Vaihingen testset

Table 4. Comparison results of multiple evaluation metrics for each model on the Vaihingen testset

Table 5. Comparison results of multiple evaluation metrics for each model on the Potsdam testset

Table 5. Comparison results of multiple evaluation metrics for each model on the Potsdam testset