To solve the problems of large parameters, high complexity, and poor detection performance of multiscale targets in the existing infrared target detection algorithms based on deep learning, a lightweight infrared target detection algorithm for multiscale targets is proposed. Based on YOLOv3, the algorithm uses the MobileNet V2 backbone network, simplified spatial pyramid structure (simSPP), anchor-free mechanism, decoupling head, and simplified positive and negative sample allocation strategies (SimOTA) to optimize the backbone, neck, and head, respectively. Finally, LMD-YOLOv3 with the model size of 6.25 M and floating-point computation of 2.14 GFLOPs was obtained. Based on the MTS-UAV data set, the mAP reached 90.5%, and on the RTX2080Ti dataset, the FPS reached 99. Compared with YOLOv3, mAP increased by 11.7%, and the model size was only 1/10 of YOLOv3.