Multi-Modal Pedestrian Detection Algorithm Based on Illumination Perception Weight Fusion

Keqi Liu; Mianmian Dong; Hui Gao; Zhigang Lü; Baoyi Guo; Min Pang

doi:10.3788/LOP222528

Laser & Optoelectronics Progress, Volume. 60, Issue 16, 1610008(2023)

Multi-Modal Pedestrian Detection Algorithm Based on Illumination Perception Weight Fusion

Keqi Liu¹, Mianmian Dong^1、*, Hui Gao¹, Zhigang Lü¹, Baoyi Guo², and Min Pang³

Author Affiliations

¹School of Electronic and Information Engineering, Xi'an Technological University, Xi'an 710021, Shaanxi, China

²Undergraduate College, Xi'an Technological University, Xi'an 710021, Shaanxi, China

³Beijing Institute of Microelectronics Technology, Beijing 100000, China

show less

Abstract Get PDF(in Chinese)

Figures & Tables(15)

Fig. 1. Structure of the proposed network

Download full size

Fig. 2. Structure of ECA mechanism module

Download full size

Fig. 3. Structure of IPWF module

Download full size

Fig. 4. SKNet structure

Download full size

Fig. 5. Visible and infrared image pair under shade trees. (a) Visible light image; (b) infrared image

Download full size

Fig. 6. MR-FPPI curves of different fusion strategies

Download full size

Fig. 7. MR-FPPI curves of different algorithms

Download full size

Fig. 8. Pedestrian target detection results in day time scenes. (a) Original annotation; (b) test result

Download full size

Fig. 9. Pedestrian target detection results in night time scenes. (a) Original annotation; (b) test result

Download full size

Fig. 10. Pedestrian target detection results on LLVIP dataset. (a) Detection results of daytime condition; (b) detection results of night condition

Download full size

Fig. 11. Pedestrian target detection results on M³FD dataset. (a) Detection results of daytime condition; (b) detection results of night condition

Download full size

Table 1. Performance comparison of different fusion policies

View table

Table 1. Performance comparison of different fusion policies

Fusion strategy	MR /%			Model size /10⁶
Fusion strategy	All the time	Day	Night	Model size /10⁶
ZJDD	15.66	17.85	11.39	305.7
IPWF	12.74	14.69	9.51	320.5
IPWF+CBAM	13.33	14.81	10.17	346.3
IPWF+SE	13.17	14.50	9.86	346.0
TIWF+ECA	12.11	13.65	9.45	317.7
IPWF+ECA	11.16	12.40	8.46	320.7

Table 2. Experimental results of loss function ablation

View table

Table 2. Experimental results of loss function ablation

Ablation strategy	$L_{c l s}$		$L_{r e g}$		$L_{I}$		MR /%
Ablation strategy	Focal loss	Cross entropy loss	Smooth L1	CIOU	Focal loss	Cross entropy loss	MR /%
Strategy 1	√		√				11.87
Strategy 2	√			√			12.06
Strategy 3		√	√				12.17
Strategy 4		√		√			12.55
Strategy 5		√	√		√		11.60
Strategy 6	√		√			√	11.16

Table 3. Performance comparison of different algorithms

View table

Table 3. Performance comparison of different algorithms

Algorithm	MR /%			Speed /（frame·s^-1）
Algorithm	All the time	Day	Night	Speed /（frame·s^-1）
ACF+T+THOG^［23］	47.32	42.57	56.17	0.27
Halfway-fusion^［4］	25.75	24.88	26.59	0.43
Fusion-RPN^［5］	18.29	19.57	16.27	0.80
IAF-RCNN^［9］	15.73	14.55	18.26	0.21
IATDNN+IAMSS^［8］	14.95	14.67	15.72	0.25
GFR^［24］	11.51	12.64	10.63	0.12
Proposed algorithm	11.16	12.40	8.46	0.08

Table 4. Performance comparison of different algorithms on LLVIP dataset and M³FD dataset

View table

Table 4. Performance comparison of different algorithms on LLVIP dataset and M³FD dataset

Algorithm	LLVIP dataset		M³FD dataset
Algorithm	MR /%	Speed /（frame·s^-1）	MR /%	Speed /（frame·s^-1）
ACF+T+THOG^［23］	59.40	0.32	51.20	0.30
Halfway-fusion^［4］	34.40	0.47	28.94	0.43
Fusion-RPN^［5］	26.15	0.81	21.10	0.80
IAF-RCNN^［9］	23.70	0.20	18.28	0.21
IATDNN+IAMSS^［8］	24.41	0.25	18.16	0.26
GFR^［24］	22.10	0.14	17.37	0.12
Proposed algorithm	20.17	0.09	16.07	0.08

Tools

Get Citation

Copy Citation Text

Keqi Liu, Mianmian Dong, Hui Gao, Zhigang Lü, Baoyi Guo, Min Pang. Multi-Modal Pedestrian Detection Algorithm Based on Illumination Perception Weight Fusion[J]. Laser & Optoelectronics Progress, 2023, 60(16): 1610008

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites