Multimodal Remote Sensing Image Classification with Small Sample Size Based on High-Level Feature Fusion

Fig. 1. Multimodal deep learning network. (a) Pixel-level fusion strategy; (b) feature-level fusion strategy; (c) high-level feature-level fusion strategy

Download full size

Fig. 2. Multimodal remote sensing image datasets. (a) High-resolution remote sensing images; (b) low-resolution remote sensing images

Download full size

Table 1. Network structure of feature extractor

View table

Table 1. Network structure of feature extractor

Network structure	Formula	Output	Feature size
cov1	X'_cov1_k=ReLU(K_cov1_k*x_i+b₁)	$\begin{matrix} {\{X'_{cov 1 k}\}}_{k = 1}^{16} \end{matrix}$	208×208
pool1	X'_pool1=f $\begin{matrix} (X'_{cov 1 k} + b_{2}) \end{matrix}$	X'_pool1	104×104
cov2	X'_cov2_k=ReLU(K_cov2_k*X'_pool1_k+b₃)	$\begin{matrix} {\{X'_{cov 2 k}\}}_{k = 1}^{16} \end{matrix}$	104×104
pool2	X'_pool2=f $\begin{matrix} (X'_{cov 2 k} + b_{4}) \end{matrix}$	X'_pool2	104×104
ful1	X'_ful1=X'_pool21±X'_pool22±…±X'_pool2104+b₅	X'_ful1	1×173056
Z'_ful1	Z'_ful1=W₁X'_ful1±W₂Y'_ful1+b₆	Z'_ful1	128×128
ful2	Z'_ful2=Z'_ful1×K_ful1+b₇	Z'_ful2	1×128

Table 2. Influence of MRSIC model parameter setting on classification accuracy
View table
Table 2. Influence of MRSIC model parameter setting on classification accuracy
Parameter 8kernels 16kernels 32kernels 8/batch 16/batch 32/batch
Time /s 1356.9 2456.2 4828.1 1431.4 2452.2 4418.32
Accuracy /% 95.3 90 92.6 50 90 82.2

Table 3. Accuracy of seven classification models

View table

Table 3. Accuracy of seven classification models

Network structure	Dataset	Amount	Accuracy /%
THRSI-LeNet	NW	1000	83.2
THRSI-LeNet	NW	500	74.6
TLRSI-LeNet	UC	1000	77.3
TLRSI-LeNet	UC	500	67.2
H&L-LeNet	NW、UC	1000	82.5
HUL-LeNet	NW、UC	1000	72.0
NanoNet	UC	500	77.1
VGG	NW、UC	500	70.4
In Fig. 1(b)	NW、UC	1000	85.0
MRSIC	NW、UC	1000	90.5
MRSIC	NW、UC	500	90.0

Table 4. Robustness verification of MRSIC
View table
Table 4. Robustness verification of MRSIC
Category MRSICaccuracy /% THRSI-LeNetaccuracy /%
bd, golf 90 74.6
te, st 87 73.8
bd, golf, te, st 75.3 64

Tools

Get Citation

Copy Citation Text

Qi He, Yao Li, Wei Song, Dongmei Huang, Shengqi He, Yanling Du. Multimodal Remote Sensing Image Classification with Small Sample Size Based on High-Level Feature Fusion[J]. Laser & Optoelectronics Progress, 2019, 56(11): 111001

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites