Online Calibration Method of Lidar‐Visual Sensor External Parameters Based on 4D Correlation Pyramid

Fig. 8. Multi-model iterative refinement calibration test results on KITTI VO dataset. (a) Initial error 1 point cloud map; (b) initial error 2 point cloud map; (c) initial error 1 prediction result; (d) initial error 2 prediction result

Download full size

Fig. 9. Multi-model iteration error distribution. (a) Initial value of translation error; (b) translation error after 1 iteration; (c) translation error after 5 iterations; (d) initial value of rotation error; (e) rotation error after 1 iteration; (f) rotation error after 5 iterations

Download full size

Fig. 10. Example of 3D color map reconstructed from KITTI VO Seq 00 by calibrated external parameters calculated by proposed algorithm

Download full size

Table 1. Single model calibration test results on KITTI VO dataset

View table

Table 1. Single model calibration test results on KITTI VO dataset

Range	Method	Translation /cm					Rotation /（°）
Range	Method	${\bar{σ}}_{t}$	$μ_{t}$	$X$	$Y$	$Z$	${\bar{σ}}_{R}$	$μ_{R}$	$R o l l$	Pitch	$Y a w$
1.5 m， 20°	LCCNet	17.83	8.210	11.84	5.169	7.613	1.235	0.480	0.170	0.676	0.594
1.5 m， 20°	Ours	15.92	4.289	3.026	5.631	4.210	1.482	0.394	0.314	0.631	0.236
1.0 m， 10°	LCCNet	6.291	2.826	2.045	4.195	2.238	0.594	0.391	0.378	0.391	0.405
1.0 m， 10°	Ours	6.589	3.610	2.552	3.269	5.010	0.625	0.152	0.105	0.229	0.122
0.5 m， 5°	LCCNet	3.915	1.529	1.267	2.212	1.107	0.414	0.324	0.309	0.330	0.334
0.5 m， 5°	Ours	3.325	1.080	1.457	0.625	1.158	0.330	0.038	0.037	0.058	0.020
0.2 m， 2°	LCCNet	2.069	0.526	0.664	0.633	0.281	0.288	0.105	0.132	0.103	0.081
0.2 m， 2°	Ours	1.503	0.427	0.218	0.652	0.411	0.165	0.106	0.075	0.087	0.155
0.1 m， 1°	LCCNet	1.588	0.361	0.243	0.380	0.459	0.163	0.030	0.030	0.019	0.040
0.1 m， 1°	Ours	0.702	0.339	0.494	0.349	0.174	0.067	0.026	0.037	0.014	0.027

Table 2. Multi-model iterative refinement calibration test results on KITTI VO dataset

View table

Table 2. Multi-model iterative refinement calibration test results on KITTI VO dataset

Result	Translation /cm					Rotation /（°）
Result	${\bar{σ}}_{t}$	$μ_{t}$	$X$	$Y$	$Z$	${\bar{σ}}_{R}$	$μ_{R}$	$R o l l$	Pitch	Yaw
Mean	0.726	0.343	0.243	0.304	0.482	0.197	0.031	0.025	0.027	0.042
Median	0.448	0.409	0.283	0.334	0.610	0.056	0.014	0.009	0.020	0.014
Standard deviation	2.607	0.237	0.199	0.124	0.388	1.654	0.036	0.029	0.020	0.059

Table 3. Comparison of translation and rotation error with initial error range of (±1.5 m, ±20°)

View table

Table 3. Comparison of translation and rotation error with initial error range of (±1.5 m, ±20°)

Method	Range	Translation /cm				Rotation /（°）
Method	Range	$X$	$Y$	$Z$	$μ_{t}$	Roll	Pitch	Yaw	$μ_{R}$
RegNet	1.5 m， 20°	7.000	7.000	4.000	6.000	0.240	0.250	0.360	0.283
CalibRCNN	1.5 m， 20°	6.200	4.300	5.400	5.300	0.199	0.640	0.446	0.428
CFNet	1.5 m， 20°	1.025	0.919	1.042	0.995	0.059	0.110	0.092	0.087
LCCNet	1.5 m， 20°	0.206	0.330	0.327	0.288	0.059	0.035	0.025	0.039
CalibBD	1.5 m， 20°	0.963	0.846	0.828	0.879	0.054	0.116	0.075	0.082
Ours	1.5 m， 20°	0.243	0.304	0.482	0.343	0.025	0.027	0.042	0.031

Table 4. Comparison of translation and rotation error with initial error range of (±0.2 m, ±20°)

View table

Table 4. Comparison of translation and rotation error with initial error range of (±0.2 m, ±20°)

Method	Range	Translation /cm				Rotation /（°）
Method	Range	$X$	$Y$	$Z$	$μ_{t}$	Roll	Pitch	Yaw	$μ_{R}$
CalibNet	0.2 m， 20°	4.200	1.600	7.220	4.340	0.150	0.900	0.180	0.410
CalibDNN	0.2 m， 20°	3.800	1.800	9.600	5.067	0.110	0.350	0.180	0.213
CFNet	0.2 m， 20°	0.463	1.230	0.802	0.832	0.028	0.125	0.105	0.086
CalibBD	0.2 m， 20°	0.325	1.066	0.878	0.756	0.021	0.136	0.084	0.080
LCCNet	0.2 m， 20°	0.167	0.176	0.265	0.203	0.056	0.054	0.018	0.043
Ours	0.2 m， 20°	0.104	0.380	0.459	0.314	0.024	0.026	0.037	0.029

Table 5. Single model calibration test results on KITTI360 dataset

View table

Table 5. Single model calibration test results on KITTI360 dataset

Range	Translation /cm				Rotation /（°）
Range	$μ_{t}$	$X$	$Y$	$Z$	$μ_{R}$	$R o l l$	Pitch	$Y a w$
1.5 m， 20°	5.229	5.141	4.911	5.634	0.843	1.621	0.373	0.534
1.0 m， 10°	2.087	3.163	1.619	1.478	0.180	0.141	0.187	0.212
0.5 m， 5°	1.303	1.209	1.752	0.949	0.148	0.081	0.111	0.251
0.2 m， 2°	0.899	1.054	0.834	0.810	0.123	0.092	0.127	0.151
0.1 m， 1°	0.318	0.246	0.411	0.299	0.065	0.041	0.072	0.082

Table 6. Comparison of calibration errors on KITTI360 dataset with different training epochs

View table

Table 6. Comparison of calibration errors on KITTI360 dataset with different training epochs

Sequence	Training 1 epoch						Training 10 epochs
	Translation /cm			Rotation /（°）			Translation /cm			Rotation /（°）
	$X$	$Y$	$Z$	Roll	Pitch	Yaw	$X$	$Y$	$Z$	Roll	Pitch	Yaw
0002	0.640	0.399	0.199	0.062	0.036	0.103	0.350	0.434	0.286	0.055	0.086	0.087
0003	1.045	1.023	0.411	0.087	0.061	0.073	0.310	0.389	0.396	0.020	0.013	0.041
0004	0.484	0.272	0.958	0.178	0.140	0.162	0.396	0.362	0.193	0.094	0.067	0.153
0005	0.371	1.077	1.427	0.177	0.603	0.688	0.231	0.599	0.411	0.033	0.035	0.022
0006	3.809	2.828	3.138	0.092	0.142	0.084	1.900	1.880	1.323	0.034	0.025	0.037
0007	0.179	0.372	0.264	0.068	0.056	0.055	0.280	0.484	0.235	0.043	0.027	0.068
0009	0.837	1.522	1.190	0.028	0.066	0.082	0.404	0.169	0.625	0.041	0.022	0.064
0010	0.844	0.581	0.714	0.960	0.089	0.050	0.321	0.074	0.493	0.033	0.019	0.019
Mean	1.026	1.009	1.038	0.207	0.149	0.162	0.524	0.549	0.495	0.044	0.037	0.061

Table 7. Error analysis of model prediction results with different loss weights

View table

Table 7. Error analysis of model prediction results with different loss weights

Loss weights	Dataset	Translation /cm				Rotation /（°）
Loss weights	Dataset	$μ_{t}$	$X$	$Y$	$Z$	$μ_{R}$	$R o l l$	Pitch	$Y a w$
0.5， 1.5， 1.0	KITTI VO	3.927	3.468	3.157	5.157	0.253	0.211	0.344	0.204
0.5， 1.5， 1.0	KITTI360	1.615	0.509	1.775	2.562	0.234	0.342	0.183	0.178
0.5， 1.0， 2.0	KITTI VO	3.610	2.552	3.269	5.010	0.152	0.105	0.229	0.122
0.5， 1.0， 2.0	KITTI360	2.087	3.163	1.619	1.478	0.180	0.141	0.187	0.212
0.5， 0.5， 3.0	KITTI VO	4.347	1.811	4.298	6.932	0.227	0.235	0.279	0.168
0.5， 0.5， 3.0	KITTI360	2.425	1.097	2.698	3.479	0.280	0.123	0.341	0.378

Tools

Get Citation

Copy Citation Text

Hui Liu, Liwen Meng, Yijian Duan, Danfeng Wu, Boru Huang, Jiachun Wu, Yanmei Meng. Online Calibration Method of Lidar‐Visual Sensor External Parameters Based on 4D Correlation Pyramid[J]. Chinese Journal of Lasers, 2024, 51(17): 1704003

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites

Paper Information

Category: Measurement and metrology

Received: Oct. 16, 2023

Accepted: Dec. 7, 2023

Published Online: Aug. 29, 2024

The Author Email: Meng Yanmei (gxu_mengyun@163.com)

DOI:10.3788/CJL231290

CSTR:32183.14.CJL231290

Topics

laser devices and laser physics

Lasers and Laser Optics

Laser physics

laser manufacturing

Instrumentation, Measurement and Metrology