Synthetic Aperture Radar and Optical Images Registration Based on Convolutional and Graph Neural Networks

Lei Liu; Yuanxiang Li; Runsheng Ni; Yuxuan Zhang; Yilin Wang; Zongcheng Zuo

doi:10.3788/AOS202242.2410002

Acta Optica Sinica, Volume. 42, Issue 24, 2410002(2022)

Synthetic Aperture Radar and Optical Images Registration Based on Convolutional and Graph Neural Networks

Lei Liu, Yuanxiang Li^*, Runsheng Ni, Yuxuan Zhang, Yilin Wang, and Zongcheng Zuo

Author Affiliations

School of Aeronautics and Astronautics, Shanghai Jiao Tong University, Shanghai 200240, China

show less

Abstract Get PDF(in Chinese)

Figures & Tables(16)

Fig. 1. Architecture of SAR and optical images registration network

Download full size

Fig. 2. Feature detection and description network

Download full size

Fig. 3. Feature point matching network

Download full size

Fig. 4. Random geometric transformation

Download full size

Fig. 5. Matching matrix. (a) Ground truth generation; (b) prediction and loss calculation

Download full size

Fig. 6. Results of feature point detection. (a) Original image; (b) ORB; (c) SIFT; (d) D2Net; (e) RIFT; (f) SuperPoint

Download full size

Fig. 7. Matching results of shift invariance experiment. (a) OS-SIFT; (b) SuperPoint; (c) D2Net; (d) CMM-Net; (e) RIFT; (f) proposed CGNet

Download full size

Fig. 8. Matching and registration results in rotation invariance experiment. (a) Matching result of 150° rotation; (b) registration result of 150° rotation; (c) matching result of 210° rotation; (d) registration result of 210° rotation

Download full size

Fig. 9. Matching results in scaling and rigid transformation. (a) Scaling scale of 0.75; (b) scaling scale of 0.6; (c) random rigid transformation 1; (d) random rigid transformation 2

Download full size

Fig. 10. $R_{S}$ varying with epoch

Download full size

Table 1. Performance evaluation of feature detector
View table
Table 1. Performance evaluation of feature detector
Index SIFT ORB RIFT D2Net SuperPoint
$N^{c}$ 158.2 196.9 178.1 158.8 221.4
$R_{r e p}$ /% 17.5 21.3 18.1 16.0 22.6

Table 2. Performance of feature matching in shift invariance experiment
View table
Table 2. Performance of feature matching in shift invariance experiment
Method $R_{S}$ $N_{C M}$ $R_{M S E}$
OS-SIFT 0.097 8.9 1.688
SuperPoint 0.454 25.9 1.705
D2Net 0.663 15.9 1.895
CMM-Net 0.880 23.3 1.928
RIFT 0.969 86.7 1.733
CGNet 0.977 171.6 1.711

Table 3. Performance of feature matching in rotation, scaling and rigid transformation experiments

View table

Table 3. Performance of feature matching in rotation, scaling and rigid transformation experiments

Transformation	Index	OS-SIFT	SuperPoint	D2Net	CMM-Net	RIFT	CGNet
Rotation	$R_{S}$	0	0.060	0.063	0.097	0.054	0.951
	$N_{C M}$		29.1	16.5	19.7	85.7	98.7
	$R_{M S E}$		1.729	1.900	1.931	1.912	1.753
Scaling	$R_{S}$	0.240	0.523	0.600	0.917	0.620	0.957
	$N_{C M}$	31.7	26.9	15.6	22.1	69.8	173.6
	$R_{M S E}$	1.748	1.877	1.885	1.953	1.869	1.811
Rigid	$R_{S}$	0.040	0.137	0.166	0.251	0.220	0.940
	$N_{C M}$	37.5	27.4	12.2	20.4	31.9	84.2
	$R_{M S E}$	1.708	1.829	1.945	1.969	1.848	1.827

Table 4. Results of ablation study

View table

Table 4. Results of ablation study

SuperPoint	Positional encoding	Number of layers of self- attention module	Number of layers of cross- attention module	$R_{S}$	$N_{C M}$	$R_{M S E}$
√				0.454	25.9	1.705
√	√			0
√	√	9		0.649	54.7	1.614
√		9	9	0.386	18.3	1.524
√	√	3	3	0.274	8.8	1.440
√	√	6	6	0.800	71.1	1.621
√	√	9	9	0.977	171.6	1.711

Table 5. Growth value of matching success rate $R_{S}$ under different epochs
View table
Table 5. Growth value of matching success rate $R_{S}$ under different epochs
$λ$ 50 100 150 200 250 300
$5$ 0.029 0.183 0.317 0.729 0.694 0.849
$10$ 0.071 0.297 0.732 0.854 0.853 0.871
$20$ 0.254 0.780 0.889 0.897 0.860 0.875

Table 6. Running time and number of parameters comparison
View table
Table 6. Running time and number of parameters comparison
Index OS-SIFT SuperPoint D2net CMM-Net RIFT CGNet
Number of parameters / $10^{6}$ 4.96 29.1 29.1 45.8
Detection time /s 12.133 0.023 0.231 0.269 5.885 0.017
Matching time /s 0.685 2.284 2.152 0.821 1.416 0.086

Tools

Get Citation

Copy Citation Text

Lei Liu, Yuanxiang Li, Runsheng Ni, Yuxuan Zhang, Yilin Wang, Zongcheng Zuo. Synthetic Aperture Radar and Optical Images Registration Based on Convolutional and Graph Neural Networks[J]. Acta Optica Sinica, 2022, 42(24): 2410002

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites