Remote Sensing Image Segmentation Based on Attention Guidance and Multi-Feature Fusion

Yinhui Zhang; Feng Zhang; Zifen He; Xiaogang Yang; Ruitao Lu; Guangchen Chen

doi:10.3788/AOS230631

Acta Optica Sinica, Volume. 43, Issue 24, 2428010(2023)

Remote Sensing Image Segmentation Based on Attention Guidance and Multi-Feature Fusion

Yinhui Zhang¹, Feng Zhang¹, Zifen He^1、*, Xiaogang Yang², Ruitao Lu², and Guangchen Chen¹

¹Faculty of Mechanical and Electrical Engineering, Kunming University of Science and Technology, Kunming 650500, Yunnan , China

²College of Missile Engineering, Rocket Force Engineering University, Xi'an 710025, Shaanxi , China

show less

Abstract Get PDF(in Chinese)

Figures & Tables(16)

Fig. 1. Architecture of the AMSNet

Download full size

Fig. 2. Category guidance channel attention module

Download full size

Fig. 3. Cross regional feature fusion module

Download full size

Fig. 4. Cloud disturbance remote sensing image

Download full size

Fig. 5. Show of dataset. (a) Original images; (b) after image corruption; (c) labels

Download full size

Fig. 6. Segmentation rendering of different networks on remote sensing image dataset of plateau region. (a) Original image; (b) label; (c) AMSNet; (d) SegNext; (e) ISANet; (f) OCNet; (g) Deeplabv3+; (h) PspNet; (i) BiseNetv2

Download full size

Fig. 7. Segmentation rendering of different networks on remote sensing image dataset of plateau area under cloud disturbance. (a) Original image; (b) label; (c) AMSNet; (d) SegNext; (e) ISANet; (f) OCNet; (g) Deeplabv3+; (h) PspNet; (i) BiseNetv2

Download full size

Fig. 8. Segmentation rendering of different networks on ISPRS Vaihingen dataset. (a) Original image; (b) label; (c) AMSNet; (d)SegNext; (e) ISANet; (f) OCNet; (g) Deeplabv3+; (h) PspNet; (i) BiseNetv2

Download full size

Table 1. Results of ablation test of each module on D_Resnet50 backbone
View table
Table 1. Results of ablation test of each module on D_Resnet50 backbone
Network MLFM CGCAM CRFFM FRM mIoU /% mPa /% mF1 /%
D_Resnet50 73.53 83.34 84.40
MLNet √ 74.80 84.31 85.26
MCNet √ √ 75.59 85.15 85.80
MCCNet √ √ √ 76.56 85.80 86.48
AMSNet √ √ √ √ 77.77 86.61 87.27

Table 2. Ablation test of different backbones
View table
Table 2. Ablation test of different backbones
Backbone mIoU /% mPa /% mF1 /% GLOPs /10⁹ Speed /（frame·s^-1）
Resnet50 72.85 83.09 83.95 256.81 7.1
D_Resnet50 77.77 86.61 87.27 546.98 6.4
Shufflenetv2 71.12 81.60 82.64 49.89 8.8
ConvNeXt 66.69 78.94 79.48 127.47 7.8

Table 3. Ablation experiment of category guidance channel attention module

View table

Table 3. Ablation experiment of category guidance channel attention module

Place for attention	mIoU /%	mPa /%	mF1 /%
Layer1	76.36	85.97	86.34
Layer2	76.44	85.85	86.40
Layer3	76.32	85.77	86.32
Layer4	75.31	84.91	85.63
Layer1+Layer2	77.77	86.61	87.27
Layer3+Layer4	75.47	85.06	85.73
Layer1+Layer2+Layer3	76.50	86.09	86.43
Layer1+Layer2+Layer3+Layer4	75.70	85.21	85.89

Table 4. Ablation experiment of feature reuse module
View table
Table 4. Ablation experiment of feature reuse module
FRM mIoU /% mPa /% mF1 /%
Origin 76.56 85.80 86.48
FRM5 76.61 85.89 86.51
FRM5+FRM7 77.35 86.38 87.00
FRM5+FRM7+FRM9 77.77 86.61 87.27
FRM5+FRM7+FRM9+FRM11 77.14 86.27 86.85

Table 5. Ablation experiment of multi-scale loss fusion module
View table
Table 5. Ablation experiment of multi-scale loss fusion module
Main loss weight $β$ mIoU /% mPa /% mF1 /%
1 0 75.61 85.37 85.86
1 0.2 76.46 86.12 86.42
1 0.4 76.73 86.23 86.60
1 0.6 77.49 86.61 87.09
1 0.8 77.22 86.32 86.91
1 1 77.77 86.61 87.27
1 1.2 77.41 86.34 87.04
1 1.4 76.61 85.50 86.50

Table 6. Comparative experiment of different networks on remote sensing image dataset of plateau region

View table

Table 6. Comparative experiment of different networks on remote sensing image dataset of plateau region

Parameter		BiseNetv2	PspNet	Deeplabv3+	OCNet	ISANet	SegNext	AMSNet
mIoU /% mPa /% mF1 /%		57.80	73.22	74.39	74.93	73.07	73.73	77.77
		70.69	83.32	83.83	84.68	83.50	84.17	86.61
		72.26	84.22	84.98	85.41	84.15	84.56	87.27
Accuracy of each class /%	River	38.28	60.56	58.25	64.60	59.61	59.38	69.97
	Lake	81.02	87.92	86.79	89.22	84.49	87.13	91.90
	Farmland	68.71	81.53	83.39	81.33	81.93	82.17	84.37
	Vegetation	49.29	67.99	69.49	70.41	67.38	69.65	70.74
	Building	49.52	67.47	72.15	68.49	69.83	68.87	72.35
	Background	59.96	73.85	76.29	75.51	75.14	75.20	77.30
FLOPs /10⁹		43	763	708	658	539	188	546
Speed /（frame·s^-1）		9.1	5.7	5.5	6.0	6.0	6.2	6.4

Table 7. Comparative experiment of different networks on remote sensing image dataset of plateau area under cloud disturbance

View table

Table 7. Comparative experiment of different networks on remote sensing image dataset of plateau area under cloud disturbance

Parameter		BiseNetv2	PspNet	Deeplabv3+	OCNet	ISANet	SegNext	AMSNet groups are 4
mIoU /% mPa /% mF1 /%		55.44	68.00	72.82	67.04	71.03	71.93	76.67
		67.75	79.06	82.66	79.49	82.34	82.84	86.03
		70.13	80.65	83.92	79.69	82.70	83.31	86.56
Accuracy of each class /%	River	35.24	56.63	57.04	53.76	55.50	57.31	68.56
	Lake	88.92	85.02	86.52	86.29	85.99	86.21	91.12
	Farmland	66.21	67.33	80.45	76.25	77.62	80.47	82.96
	Vegetation	47.68	65.48	68.57	63.16	67.47	68.03	69.87
	Building	43.83	63.33	68.77	54.04	67.63	65.76	71.05
	Background	58.77	70.23	75.53	68.73	71.95	73.80	76.48
FLOPs /10⁹		43	763	708	658	539	188	669
Speed /（frame·s^-1）		9.1	5.7	5.6	6.0	6.0	6.2	6.0

Table 8. Comparative experiment of different networks on ISPRS Vaihingen dataset
View table
Table 8. Comparative experiment of different networks on ISPRS Vaihingen dataset
Parameter BiseNetv2 PspNet Deeplabv3+ OCNet ISANet SegNext
AMSNet
groups are 4
mIoU /% 59.00 66.82 68.44 66.34 63.37 70.58 71.91
mPa /% 70.65 77.73 77.36 76.96 77.51 80.17 80.92
mF1 /% 71.45 79.06 80.08 78.65 76.97 81.61 82.86

Tools

Get Citation

Copy Citation Text

Yinhui Zhang, Feng Zhang, Zifen He, Xiaogang Yang, Ruitao Lu, Guangchen Chen. Remote Sensing Image Segmentation Based on Attention Guidance and Multi-Feature Fusion[J]. Acta Optica Sinica, 2023, 43(24): 2428010

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites

Paper Information

Category: Remote Sensing and Sensors

Received: Mar. 6, 2023

Accepted: Apr. 24, 2023

Published Online: Dec. 8, 2023

The Author Email: He Zifen (zyhhzf1998@168.com)

DOI:10.3788/AOS230631

Topics

laser devices and laser physics

Lasers and Laser Optics

Laser physics

laser manufacturing

Instrumentation, Measurement and Metrology

Table 1. Results of ablation test of each module on D_Resnet50 backbone

Table 1. Results of ablation test of each module on D_Resnet50 backbone

Table 2. Ablation test of different backbones

Table 2. Ablation test of different backbones

Table 3. Ablation experiment of category guidance channel attention module

Table 3. Ablation experiment of category guidance channel attention module

Table 4. Ablation experiment of feature reuse module

Table 4. Ablation experiment of feature reuse module

Table 5. Ablation experiment of multi-scale loss fusion module

Table 5. Ablation experiment of multi-scale loss fusion module

Table 6. Comparative experiment of different networks on remote sensing image dataset of plateau region

Table 6. Comparative experiment of different networks on remote sensing image dataset of plateau region

Table 7. Comparative experiment of different networks on remote sensing image dataset of plateau area under cloud disturbance

Table 7. Comparative experiment of different networks on remote sensing image dataset of plateau area under cloud disturbance

Table 8. Comparative experiment of different networks on ISPRS Vaihingen dataset

Table 8. Comparative experiment of different networks on ISPRS Vaihingen dataset