Transfer Learning Based Mixture of Experts Classification Model for High-Resolution Remote Sensing Scene Classification

Xi Gong; Zhanlong Chen; Liang Wu; Zhong Xie; Yongyang Xu

doi:10.3788/AOS202141.2301003

Acta Optica Sinica, Volume. 41, Issue 23, 2301003(2021)

Transfer Learning Based Mixture of Experts Classification Model for High-Resolution Remote Sensing Scene Classification

Xi Gong¹, Zhanlong Chen^1,2, Liang Wu^1,2, Zhong Xie^1,2、*, and Yongyang Xu^1,2

Author Affiliations

¹School of Geography and Information Engineering, China University of Geosciences, Wuhan, Hubei 430074, China

²National Engineering Research Center of Geographic Information System, Wuhan, Hubei 430074, China

show less

Abstract Get PDF(in Chinese)

Figures & Tables(16)

Fig. 1. Flow chart of TLMoE

Download full size

Fig. 2. Transfer learning process of expert network

Download full size

Fig. 3. Training sample filter for expert networks

Download full size

Fig. 4. Image examples of remote sensing scenes. (a) UCM dataset; (b) SIRI dataset; (c) RSSCN7 dataset

Download full size

Fig. 5. Classification confusion matrix of TLMoE-VGG19 on UCM dataset

Download full size

Fig. 6. Classification confusion matrix of TLMoE-VGG19 on SIRI dataset

Download full size

Fig. 7. Classification confusion matrix of TLMoE-VGG19 on RSSCN7 dataset

Download full size

Fig. 8. Time consumption comparison before and after the combination of channels and pre-trained CNN in TLMoE. (a) VGG19; (b) Resnet50

Download full size

Fig. 9. Comparison of different features on the 3 datasets by 2-dimensional feature visualization

Download full size

Table 1. Structure comparison between VGG19 and Resnet50

View table

Table 1. Structure comparison between VGG19 and Resnet50

No.	Layer group	VGG19		Resnet50
No.	Layer group	Layer number	Feature size	Layer number	Feature size
1	conv1	2	64×224×224	1	64×112×112
2	conv2	2	128×112×112	9	256×56×56
3	conv3	4	256×56×56	12	512×28×28
4	conv4	4	512×28×28	18	1024×14×14
5	conv5	4	512×14×14	9	2048×7×7
6	FC/GAP	2	4096	1	2048
7	output FC	1	1000	1	1000

Table 2. Classification accuracy comparison on the UCM dataset

View table

Table 2. Classification accuracy comparison on the UCM dataset

No.	Method	Accuracy /%
1	RF^[20]	44.77
2	SIFT+BoVW^[21]	76.81
3	SIFT+SPMK^[22]	75.29
4	VGG19 (training from scratch)	83.48
5	Resnet50 (training from scratch)	85.71
6	DCT-CNN^[1]	95.76
7	Pre-trained VGG19 features+SVM	94.29
8	Pre-trained Resnet50 features+SVM	97.14
9	GLDFB^[5]	97.62
10	TLMoE-VGG19	98.10
11	TLMoE-Resnet50	98.33

Table 3. Classification accuracy comparison on the confusing classes of UCM datasetunit: %
View table
Table 3. Classification accuracy comparison on the confusing classes of UCM datasetunit: %
Type Road type Building type Other
Class Freeway Intersection Overpass Buildings Lenseresidential Mediumresidential Tenniscourt
GLDFB(VGG19) 100 100 95 90 95 95 90
TLMoE-VGG19 100 100 100 90 95 100 95

Table 4. Classification accuracy comparison on the SIRI dataset

View table

Table 4. Classification accuracy comparison on the SIRI dataset

No.	Method	Accuracy /%
1	RF^[20]	49.90
2	SIFT+BoVW^[21]	75.63
3	SIFT+SPMK^[22]	77.69±1.01
4	VGG19 (training from scratch)	86.13
5	Resnet50 (training from scratch)	89.26
6	MCNN^[23]	93.75±1.13
7	Pre-trained VGG19 features+SVM	94.79
8	Pre-trained Resnet50 features+SVM	96.25
9	GLDFB^[5]	96.67
10	TLMoE-VGG19	97.29
11	TLMoE-Resnet50	97.50

Table 5. Classification accuracy comparison on the RSSCN7 dataset

View table

Table 5. Classification accuracy comparison on the RSSCN7 dataset

No.	Method	Accuracy /%
1	RF^[20]	55.43
2	VGG19 (training from scratch)	82.50
3	Resnet50 (training from scratch)	81.70
4	Deep filter bank^[24]	90.04±0.6
5	Pre-trained VGG19 features+SVM	91.93
6	Pre-trained Resnet50 features+SVM	89.92
7	TLMoE-VGG19	93.21
8	TLMoE-Resnet50	93.29

Table 6. Classification accuracy comparison between TLMoE channels

View table

Table 6. Classification accuracy comparison between TLMoE channels

No.	Channel	Accuracy (pre-trained VGG19) /%			Accuracy (pre-trained Resnet50) /%
No.	Channel	UCM	SIRI	RSSCN7	UCM	SIRI	RSSCN7
1	Pre-judged channel	94.60	93.13	87.50	97.62	96.25	89.21
2	Expert channel	93.81	96.04	92.14	96.67	97.08	92.93
3	(F_L(s), X(s))-SVM	94.29	94.79	91.93	97.14	96.25	89.92
4	TLMoE	98.10	97.29	93.21	98.33	97.50	93.29

Table 7. Classification accuracy comparison of several kinds of features

View table

Table 7. Classification accuracy comparison of several kinds of features

No.	Feature	Accuracy /%
No.	Feature	UCM	SIRI	RSSCN7
1	HOG	52.14	44.79	35.79
2	SIFT	58.33	53.96	54.14
3	LBP	31.43	46.25	56.14
4	X(s)-VGG19	93.33	94.38	90.71
5	Expert channel-VGG19	93.81	96.04	92.14
6	X(s)-Resnet50	95.48	96.46	92.14
7	Expert channel-Resnet50	96.67	97.08	92.93

Tools

Get Citation

Copy Citation Text

Xi Gong, Zhanlong Chen, Liang Wu, Zhong Xie, Yongyang Xu. Transfer Learning Based Mixture of Experts Classification Model for High-Resolution Remote Sensing Scene Classification[J]. Acta Optica Sinica, 2021, 41(23): 2301003

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites