Multimodal LiDAR Enhancement Algorithm Based on Multiscale Features

Algorithm	Modality	mAP	NDS	Car	Truck	C.V.	Bus	Trailer	Barrier	Motor	Bike	Ped.	T.C.
HoP^［30］	C	62.4	68.5	77.8	53.8	35.2	52.2	56.9	74.2	70.0	55.8	68.2	79.4
SparseBEV^［31］	C	60.3	67.5	76.3	49.2	35.6	44.2	53.4	76.8	64.0	53.7	65.8	83.7
FSTR-L^［32］	L	70.2	73.6	87.1	58.4	32.9	69.3	60.6	80.6	80.2	57.0	90.0	85.7
FocalFormer3D^［33］	L	68.7	72.6	87.2	57.0	34.4	69.6	64.9	77.8	76.2	49.6	88.2	82.3
TransFusion-L^［26］	L	65.5	70.2	86.2	56.7	28.2	66.3	58.8	78.2	68.3	44.2	86.1	82.0
TransFusion^［26］	C+L	68.9	71.7	87.1	60.0	33.1	68.3	60.8	78.1	73.6	52.9	88.4	86.7
BEVFusion^［34］	C+L	71.3	73.3	88.5	63.1	38.1	72.0	64.7	78.3	75.2	56.5	90.0	86.5
EA-LSS^［35］	C+L	76.6	77.6	90.2	67.1	43.9	76.7	69.1	84.1	85.9	66.6	91.3	91.2
FusionFormer-base^［36］	C+L	72.6	75.1	89.8	62.0	40.3	69.8	63.0	77.4	83.0	64.6	90.4	86.0
MegFusion^［37］	C+L	75.3	77.0	89.8	66.3	41.7	74.5	68.4	82.1	84.0	65.9	91.6	88.8
FocalFormer3D-F^［33］	C+L	72.4	74.5	88.7	60.1	33.7	71.3	67.9	78.8	80.6	62.8	89.9	86.5
Proposed algorithm	C+L	77.1	78.6	90.6	67.5	44.4	76.8	69.9	84.4	86.2	67.5	91.5	91.5

Table 2. Generalization ability test of proposed model
View table
Table 2. Generalization ability test of proposed model
Modality PointPillar CenterPoint TransFusion-L
Camera LiDAR mAP NDS mAP NDS mAP NDS
√ 57.1 61.8 60.3 65.3 65.5 70.2
√ √ 70.1 72.6 71.7 73.0 77.1 78.6

Table 3. Robustness testing under different weather and lighting conditions

View table

Table 3. Robustness testing under different weather and lighting conditions

Algorithm	Modality	Sunny		Rainy		Day		Night
Algorithm	Modality	mAP	NDS	mAP	NDS	mAP	NDS	mAP	NDS
SparseBEV	C	60.6	67.8	58.5	63.4	60.2	66.9	50.3	54.7
FSTR-L	L	70.1	73.3	60.1	65.6	69.8	72.6	67.4	70.3
TransFusion-L	L	66.6	70.1	52.3	58.9	65.4	70.5	60.8	65.5
EA-LSS	C+L	76.4	77.5	68.7	72.5	76.1	77.7	72.0	75.2
Proposed algorithm	C+L	76.8	77.9	69.6	73.0	76.5	77.7	72.3	75.8

Table 4. Robustness testing in case of LiDAR migration

View table

Table 4. Robustness testing in case of LiDAR migration

Discrepancy /m	FSTR-L		TransFusion-L		EA-LSS		Proposed algorithm
Discrepancy /m	mAP	NDS	mAP	NDS	mAP	NDS	mAP	NDS
0.0	69.8	72.6	65.4	70.5	76.1	77.7	76.8	77.9
0.2	69.4	72.3	65.0	70.0	76.0	77.6	76.7	77.8
0.4	68.9	71.9	64.1	69.8	75.7	77.4	76.7	77.6
0.8	68.4	71.6	63.5	68.8	75.5	77.1	76.5	77.6

Table 5. Ablation experiments
View table
Table 5. Ablation experiments
Self-attention Q.I T.L Epoch mAP NDS
SSA √ √ 25 69.6 73.3
SSA √ 45 68.7 72.5
MHSA √ √ 25 62.7 66.1
SSA √ 25 66.5 70.7
MHSA 45 59.8 63.4

Tools

Get Citation

Copy Citation Text

Yikai Luo, Linyuan He, shiping Ma. Multimodal LiDAR Enhancement Algorithm Based on Multiscale Features[J]. Laser & Optoelectronics Progress, 2024, 61(18): 1815006

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites