Armored vehicles serve as crucial ground combat equipment, playing an irreplaceable role in urban attacks, defense, beach landings, and various other operations. Hence, researching armored vehicle detection technology in complex ground environments holds significant importance for accurate battlefield perception, situational awareness, precise fire targeting, and seizing battlefield opportunities. Existing image-based detection methods for armored vehicles primarily utilize visible or infrared images. Visible images often struggle to effectively handle interference from similar backgrounds, smoke, dust, and camouflage in complex ground battlefield environments. While infrared images can overcome some limitations of visible images, they often lack sufficient texture and color information. Therefore, integrating visible and infrared images and leveraging their complementary characteristics can enhance feature representation and help elevate the detection capabilities of armored vehicles in complex ground battlefield environments.

To address the challenge of detecting armored vehicles in complex land environments, we put forward a visible-infrared armored vehicle detection method that leverages feature alignment and region-based image quality guided fusion. Firstly, we enhance the YOLOv8 object detection method, a state-of-the-art one-stage anchor-free approach, by incorporating a backbone network for infrared feature extraction. This expansion results in a dual-stream architecture for enhanced performance. During the extraction of infrared features, a feature alignment module is introduced built on deformable convolutional networks. This module effectively aligns infrared features, addressing issues caused by misalignment in images. To fully utilize the complementary nature of visible and infrared images, we design a regional image quality guided fusion module for integrating features from both modalities. The regional image quality guided fusion module assesses the quality of multi-scale visible and infrared features extracted by the dual-stream feature extraction network. It generates quality matrices for both visible and infrared features, which are then processed using the Softmax function to obtain weight matrices. These weight matrices are used to combine the two modal features, resulting in a fusion feature achieved through element-wise addition or channel concatenation. Finally, the fusion feature passes through the Neck and the detection head to produce the detection results.

Experimental verification is conducted using self-built visible-infrared armored vehicle image datasets as well as publicly available FPR-aligned datasets. To simulate position shifts, the infrared image is moved along the x and y axes. The experimental results demonstrate that our feature alignment module exhibits a more pronounced effect with increasing offset (Table 5), effectively mitigating the adverse effects of position offset and enhancing the model’s robustness. Furthermore, the regional image quality guided fusion module offers improved assessment of regional image quality, fully leveraging the complementarity of the two modal features and attenuating the impact of disturbed regional image features during cross-modal feature fusion (Fig. 7). In comparison to object detection methods that solely utilize visible images, our method has shown improvements in mAP and mAP50 by 1.9% and 3.5%, respectively (Table 7). Additionally, our method demonstrates enhanced capability in addressing challenges such as smoke shielding, interference from similar ground objects, and slight dust shielding, thereby elevating the level of armored vehicle detection (Fig. 8).

We propose a visible-infrared detection method for armored vehicles based on feature alignment and regional image quality guided fusion to address challenges such as position deviation and varying importance of visible light features across different spatial locations in complex ground environments. The method integrates a feature alignment module, utilizing feasible variable convolution within a two-stream feature extraction network, to align infrared images and strengthen model robustness against unaligned image pairs. Additionally, we design a regional image quality guided fusion module, leveraging semantic label information to train a network for evaluating regional image quality and using the resulting feature quality matrix to guide the fusion of visible and infrared features. Experimental evaluations are conducted on a self-built visible-infrared armored vehicle image dataset, demonstrating that our proposed method outperforms state-of-the-art object detection methods. By effectively leveraging the complementarity of visible and infrared images, this method significantly improves the accuracy and success rate of armored vehicle detection in complex ground environments.