To overcome the limitations of low infrared image resolution, poor texture information, and blurry details of small distant targets, we propose the YOLOv5s-CA algorithm, which modifies the YOLOv5s network structure from the perspective of the attention mechanism. The algorithm adds coordinate attention to the YOLOv5s model, enabling it to focus not only on the location information between channels but also on long-range spatial location information. By integrating this additional attention mechanism into the YOLOv5s network architecture and comparing it with the original YOLOv5s model, this study demonstrates the advantages of the algorithm in both speed and accuracy. Experimental results on a homemade infrared dataset for open-pit mining areas show that the model's mean average precision (mAP) reaches 0.948—1.4% higher than the original model—with an inference speed of 3.3 ms on a GeForce 2080 Ti device. Compared with other leading algorithms, this algorithm can maintain its speed while achieving high detection accuracy for infrared targets.