Vehicle infrared image target detection is an important way of road environment perception for autonomous driving. However, existing vehicle infrared image target detection algorithms have defects, such as low memory utilization, complex calculation and low detection accuracy. In order to solve the above problems, an improved YOLOv5s lightweight target detection algorithm is proposed. Firstly, the C3Ghost and Ghost modules are introduced into the YOLOv5s detection network to reduce network complexity. Secondly, the αIoU loss function is introduced to improve the positioning accuracy of the target and the networks training efficiency. Then, the subsampling rate of the network structure is reduced and the KMeans clustering algorithm is used to optimize the prior anchor size to improve the ability to detect of small targets. Finally, coordinate attention and spatial depth convolution modules are respectively introduced into the Backbone and Neck to further optimize the model and improve the feature extraction of the model. The experimental results show that compared with the original YOLOv5s algorithm, the improved algorithm can compress the model size by 78.1%, reduce the number of parameters and Giga Floating-point Operations Per Second by 84.5% and 40.5% respectively, and improve the mean average precision and detection speed by 4.2% and 10.9%, respectively.