ing at the real-time request of the infrared detection system for target detection, we propose a method for fast detection of infrared targets based on key points. Taking the target center as the key point of target detection, we first design a lightweight feature extraction network. Then, we design a corresponding feature fusion network using the spatial and semantic information of features at different levels combined with the characteristics of small infrared targets. Finally, the prediction of target category, location and size is realized. The model is comparatively tested on the self-built aerial infrared target dataset. Compared with the classic detection models such as YOLOv3, the detection speed is greatly improved and the detection accuracy is only slightly reduced. Compared with the same type of fast detection model, Tiny-YOLOv3, the detection accuracy increases by 8.9% and the detection speed running on the central processing unit (CPU)increases by 13.9 ms/frame under the condition that the model size is compressed to 23.39% of Tiny-YOLOv3's size. The detection performance is significantly improved and the effectiveness of the method is confirmed.