Method of Visible-Infrared Armored Vehicle Detection Based on Feature Alignment and Regional Image Quality Guided Fusion

Fig. 6. Infrared feature visualization images before and after the FAM. (a) Input infrared image; (b) infrared feature visualization images before FAM processing; (c) infrared feature visualization images after FAM processing

Download full size

Fig. 7. Heat maps of multiscale quality matrix. (a) Input image pair; (b) small scale quality matrix heat maps; (c) medium scale quality matrix heat maps; (d) large scale quality matrix heat maps

Download full size

Fig. 8. Test results for different methods on partial test set. (a) No interference; (b) dust interference; (c) black smoke interference

Download full size

Table 1. VTAV image dataset information
View table
Table 1. VTAV image dataset information
Dataset name Number of image pairs Number of armoured vehicles
Training set 4862 11633
Test set 1171 4245
Total 5853 15878

Table 2. Performance comparison between single-light image object detection method and dual-light image object detection method
View table
Table 2. Performance comparison between single-light image object detection method and dual-light image object detection method
Method Input mAP50 /% mAP /%
YOLOv8-s-IR IR 57.9 24.9
YOLOv8-s-RGB RGB 89.3 56.3
YOLOv8-s-add RGB+IR 91.8 57.1
YOLOv8-s-concat RGB+IR 91.7 57.2

Table 3. Effect of position shift on the performance of YOLOv8-s-add

View table

Table 3. Effect of position shift on the performance of YOLOv8-s-add

$Δ y$ /pixel	mAP /%
$Δ y$ /pixel	$Δ x$ =-8 pixel	$Δ x$ =-4 pixel	$Δ x$ =-2 pixel	$Δ x$ =0 pixel	$Δ x$ =2 pixel	$Δ x$ =4 pixel	$Δ x$ =8 pixel
-8	55.4（-1.7）	55.6（-1.5）	55.6（-1.5）	55.6（-1.5）	55.6（-1.5）	55.6（-1.5）	55.5（-1.6）
-4	55.9（-1.2）	56.2（-0.9）	56.2（-0.9）	56.2（-0.9）	56.2（-0.9）	56.2（-0.9）	56.0（-1.1）
-2	56.3（-0.8）	56.7（-0.4）	56.7（-0.4）	56.8（-0.3）	56.8（-0.3）	56.6（-0.5）	56.2（-0.9）
0	56.5（-0.6）	56.8（-0.3）	57.0（-0.1）	57.1（-0）	57.0（-0.1）	56.8（-0.3）	56.2（-0.9）
2	56.5（-0.6）	56.8（-0.3）	56.9（-0.2）	56.8（-0.3）	56.8（-0.3）	56.7（-0.4）	56.0（-1.1）
4	56.1（-1.0）	56.4（-0.7）	56.5（-0.6）	56.5（-0.6）	56.4（-0.7）	56.2（-0.9）	55.8（-1.3）
8	55.4（-1.7）	55.5（-1.6）	55.4（-1.7）	55.5（-1.6）	55.5（-1.6）	55.5（-1.6）	55.2（-1.9）

Table 4. Effect of position shift on the performance of YOLOv8-s-add+FAM

View table

Table 4. Effect of position shift on the performance of YOLOv8-s-add+FAM

$Δ y$ /pixel	mAP /%
$Δ y$ /pixel	$Δ x$ =-8 pixel	$Δ x$ =-4 pixel	$Δ x$ =-2 pixel	$Δ x$ =0 pixel	$Δ x$ =2 pixel	$Δ x$ =4 pixel	$Δ x$ =8 pixel
-8	56.6（-0.8）	56.8（-0.6）	56.8（-0.6）	56.8（-0.6）	56.9（-0.5）	56.8（-0.6）	56.8（-0.6）
-4	57.0（-0.4）	57.0（-0.4）	57.0（-0.4）	57.1（-0.3）	57.1（-0.3）	57.1（-0.3）	57.0（-1.4）
-2	57.1（-0.3）	57.2（-0.2）	57.2（-0.2）	57.2（-0.2）	57.3（-0.1）	57.2（-0.2）	57.1（-0.3）
0	57.1（-0.3）	57.3（-0.1）	57.4（-0）	57.4（-0）	57.4（-0）	57.3（-0.1）	57.1（-0.3）
2	57.2（-0.4）	57.3（-0.1）	57.3（-0.1）	57.4（-0）	57.3（-0.1）	57.2（-0.2）	57.1（-0.3）
4	57.1（-0.3）	57.3（-0.1）	57.3（-0.1）	57.3（-0.1）	57.2（-0.2）	57.2（-0.2）	57.0（-0.4）
8	56.9（-0.5）	57.0（-0.4）	57.0（-0.4）	57.0（-0.4）	57.0（-0.4）	56.8（-0.6）	56.8（-0.6）

Table 5. Performance comparison between YOLOv8-s-add+FAM and YOLOv8-s-add

View table

Table 5. Performance comparison between YOLOv8-s-add+FAM and YOLOv8-s-add

$Δ y$ /pixel	Difference of mAP /%
$Δ y$ /pixel	$Δ x$ =-8 pixel	$Δ x$ =-4 pixel	$Δ x$ =-2 pixel	$Δ x$ =0 pixel	$Δ x$ =2 pixel	$Δ x$ =4 pixel	$Δ x$ =8 pixel
-8	1.2	1.3	1.2	1.2	1.3	1.2	1.3
-4	1.1	0.8	0.8	0.9	0.9	0.9	1.0
-2	0.8	0.5	0.5	0.4	0.5	0.6	0.9
0	0.6	0.5	0.3	0.3	0.4	0.5	0.9
2	0.7	0.5	0.4	0.6	0.5	0.5	1.1
4	1.0	0.9	0.8	0.8	0.8	1.0	1.2
8	1.5	1.5	1.6	1.5	1.5	1.3	1.6

Table 6. Performance of the RIQGFM

View table

Table 6. Performance of the RIQGFM

Method	Input	RIQGFM	mAP /%
YOLOv8-s-add	RGB+IR	—	57.1
YOLOv8-s-add	RGB+IR	√	57.8 （0.7）
YOLOv8-s-concat	RGB+IR	—	57.2
YOLOv8-s-concat	RGB+IR	√	57.8 （0.6）
YOLOv8-s-add+FAM	RGB+IR	—	57.4
YOLOv8-s-add+FAM	RGB+IR	√	58.2 （0.8）
YOLOv8-s-concat+FAM	RGB+IR	—	57.4
YOLOv8-s-concat+FAM	RGB+IR	√	58.2 （0.8）

Table 7. Performance comparison between the proposed method and the current advanced target detection methods

View table

Table 7. Performance comparison between the proposed method and the current advanced target detection methods

Method	Input	mAP50 /%	mAP /%
YOLOv5-n	RGB	87.6	54.9
YOLOv5-s	RGB	88.3	56.2
YOLOX-tiny	RGB	85.3	51.4
YOLOX-s	RGB	89.7	55.6
YOLOv7-tiny	RGB	92.3	55.8
PPYOLOE + s	RGB	91.4	55.6
Rtmdet-tiny	RGB	87.2	55.4
YOLOv8-n	RGB	88.6	55.3
YOLOv8-s-RGB	RGB	89.3	56.3
YOLOv8-s-add	RGB+IR	91.8	57.1
YOLOv8-s-concat	RGB+IR	91.7	57.2
Ours	RGB+IR	92.8	58.2

Table 8. Performance comparison between the proposed method and the dual-modal detection method

View table

Table 8. Performance comparison between the proposed method and the dual-modal detection method

Method	Input	mAP50 /%	mAP75 /%	mAP /%
CFR	RGB+IR	72.4	—	—
GAFF	RGB+IR	72.9	30.9	37.3
CAPTM	RGB+IR	73.2	—	—
CFT	RGB+IR	78.3	—	40.2
ICAFusion	RGB+IR	79.2	36.9	41.1
YOLOv8-s-add	RGB+IR	77.8	38.7	42.5
YOLOv8-s-concat	RGB+IR	78.8	39.3	42.5
Ours	RGB+IR	79.1	41.1	43.6

Tools

Get Citation

Copy Citation Text

Jie Zhang, Tianqing Chang, Libin Guo, Bin Han, Lei Zhang. Method of Visible-Infrared Armored Vehicle Detection Based on Feature Alignment and Regional Image Quality Guided Fusion[J]. Acta Optica Sinica, 2024, 44(13): 1315001

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites

Paper Information

Category: Machine Vision

Received: Feb. 28, 2024

Accepted: Mar. 24, 2024

Published Online: Jul. 17, 2024

The Author Email: Zhang Jie (zjwhy_8@163.com), Chang Tianqing (changtianqing@263.net), Guo Libin (binexe@126.com)

DOI:10.3788/AOS240664

CSTR:32393.14.AOS240664

Topics

laser devices and laser physics

Lasers and Laser Optics

Laser physics

laser manufacturing

Instrumentation, Measurement and Metrology

Table 1. VTAV image dataset information

Table 1. VTAV image dataset information

Table 2. Performance comparison between single-light image object detection method and dual-light image object detection method

Table 2. Performance comparison between single-light image object detection method and dual-light image object detection method

Table 3. Effect of position shift on the performance of YOLOv8-s-add

Table 3. Effect of position shift on the performance of YOLOv8-s-add

Table 4. Effect of position shift on the performance of YOLOv8-s-add+FAM

Table 4. Effect of position shift on the performance of YOLOv8-s-add+FAM

Table 5. Performance comparison between YOLOv8-s-add+FAM and YOLOv8-s-add

Table 5. Performance comparison between YOLOv8-s-add+FAM and YOLOv8-s-add

Table 6. Performance of the RIQGFM

Table 6. Performance of the RIQGFM

Table 7. Performance comparison between the proposed method and the current advanced target detection methods

Table 7. Performance comparison between the proposed method and the current advanced target detection methods

Table 8. Performance comparison between the proposed method and the dual-modal detection method

Table 8. Performance comparison between the proposed method and the dual-modal detection method