Classification of High-Resolution Remote Sensing Image Based on Swin Transformer and Convolutional Neural Network

Xiaoying He; Weiming Xu; Kaixiang Pan; Juan Wang; Ziwei Li

doi:10.3788/LOP232003

Laser & Optoelectronics Progress, Volume. 61, Issue 14, 1428002(2024)

Classification of High-Resolution Remote Sensing Image Based on Swin Transformer and Convolutional Neural Network

Xiaoying He^1,2,3, Weiming Xu^1,2,3、*, Kaixiang Pan^1,2,3, Juan Wang^1,2,3, and Ziwei Li^1,2,3

Author Affiliations

¹The Academy of Digital China, Fuzhou University, Fuzhou 350108, Fujian , China

²Key Laboratory of Spatial Data Mining & Information Sharing, Ministry of Education, Fuzhou University, Fuzhou 350002, Fujian , China

³National Engineering Research Center of Geospatial Information Technology, Fuzhou University, Fuzhou 350002, Fujian , China

show less

Abstract Get PDF(in Chinese)

Figures & Tables(15)

Fig. 1. Overall architecture of SRAU-Net

Download full size

Fig. 2. Standard Transformer block and Swin Transformer block. (a) Standard Transformer block; (b) Swin Transformer block

Download full size

Fig. 3. Convolution block and residual block. (a) Convolution block; (b) residual block

Download full size

Fig. 4. FFM

Download full size

Fig. 5. FEM

Download full size

Fig. 6. Visualization of segmentation results of different models on Vaihingen dataset

Download full size

Fig. 7. Visualization of segmentation results of different models on GID dataset

Download full size

Fig. 8. Visualization of the ablation experiment results. (a) baseline; (b) baseline+residual block; (c) baseline+residual block+FFM; (d) baseline+residual block+FEM; (e) baseline+residual block+FFM+FEM

Download full size

Table 1. Class distribution and weight calculation of Vaihingen dataset

View table

Table 1. Class distribution and weight calculation of Vaihingen dataset

Class name	Number of pixels	Class frequency	Weight
Impervious surface	46792757	0.2780	0.7939
Building	43779851	0.2601	0.8486
Low vegetation	35766767	0.2125	1.0387
Tree	38534748	0.2290	0.9641
Car	2096078	0.0125	17.7239
Clutter/background	1317670	0.0078	28.1943

Table 2. Class distribution and weight calculation of GID dataset

View table

Table 2. Class distribution and weight calculation of GID dataset

Class name	Number of pixels	Class frequency	Weight
Building	651080927	0.0899	1.0996
Water	780799058	0.1079	0.9169
Forest	277330405	0.0383	2.5815
Farmland	2222929336	0.3071	0.3221
Meadow	144943831	0.0200	4.9394
Others	3162486239	0.4368	0.2264

Table 3. Confusion matrix
View table
Table 3. Confusion matrix
Actual Predict
Positive Negative
Positive True positive（N_TP） False positive（N_FP）
Negative False positive（N_FP） True negative（N_TN）

Table 4. Comparison of segmentation results of different models on Vaihingen dataset

View table

Table 4. Comparison of segmentation results of different models on Vaihingen dataset

Method	IoU /%					MIoU /%	OA /%	F1 /%
Method	Impervious surface	Building	Low vegetation	Tree	Car	MIoU /%	OA /%	F1 /%
FCN	65.96	86.34	67.09	75.63	67.17	72.44	85.67	76.56
U-Net	80.40	88.07	70.32	77.78	74.16	78.15	89.57	82.59
DeeplabV3+	81.15	88.62	69.97	78.32	73.95	78.40	89.39	82.93
TransUNet	81.26	89.79	71.01	78.49	74.95	79.10	90.02	83.64
SRAU-Net	83.88	90.97	73.22	80.95	77.71	81.35	92.60	86.90

Table 5. Comparison of segmentation results of different models on GID dataset

View table

Table 5. Comparison of segmentation results of different models on GID dataset

Method	IoU /%					MIoU /%	OA /%	F1 /%
Method	Building	Water	Forest	Farmland	Meadow	MIoU /%	OA /%	F1 /%
FCN	47.03	72.50	46.72	49.11	43.08	51.69	70.85	65.11
U-Net	62.21	81.22	51.07	68.79	64.08	65.47	82.80	69.05
DeeplabV3+	63.22	79.49	53.23	67.53	60.93	64.88	80.66	67.14
TransUNet	63.03	80.35	55.02	68.93	64.24	66.31	82.91	69.12
SRAU-Net	64.58	80.82	55.73	70.11	65.52	67.35	84.26	71.84

Table 6. Ablation experiment setting of SRAU-Net
View table
Table 6. Ablation experiment setting of SRAU-Net
Model name Baseline Residual block FFM FEM
（a） √
（b） √ √
（c） √ √ √
（d） √ √ √
（e） √ √ √ √

Table 7. Result of the ablation experiment on Vaihingen dataset

View table

Table 7. Result of the ablation experiment on Vaihingen dataset

Method	IoU /%					MIoU /%	OA /%	F1 /%
Method	Impervious surface	Building	Low vegetation	Tree	Car	MIoU /%	OA /%	F1 /%
（a）	82.66	89.37	70.07	77.03	75.12	78.85	90.03	85.55
（b）	83.68	90.09	73.02	80.67	76.33	80.76	91.25	86.38
（c）	83.80	90.82	73.05	80.69	76.36	80.94	92.35	86.62
（d）	83.78	90.87	73.14	80.72	76.43	80.98	92.37	86.68
（e）	83.88	90.97	73.22	80.95	77.71	81.35	92.60	86.90

Tools

Get Citation

Copy Citation Text

Xiaoying He, Weiming Xu, Kaixiang Pan, Juan Wang, Ziwei Li. Classification of High-Resolution Remote Sensing Image Based on Swin Transformer and Convolutional Neural Network[J]. Laser & Optoelectronics Progress, 2024, 61(14): 1428002

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites