Aiming at the problems of low target detection accuracy for small-sized drones and the deep network has a large number of parameters and a high memory footprint, a drone detection method based on improved YOLOv5 was proposed. Firstly, the number of YOLOv5 multi-scale prediction layers was adjusted, and the redundant network layers was cut, which effectively reduced the amount of network parameters, and improved the speed of drone detection. Secondly, multiple parallel atrous convolutions with different sampling rates were introduced into the feature extraction stage to enhance the ability of multi-scale detail feature extraction of small targets. Finally, the attention mechanism was introduced into the multi-scale feature fusion stage to enhance the feature expression ability of small targets by Fusion of shallow features and deep features were channel-weighted. The experimental results illustrate that the improved YOLOv5 model achieves 99.02% mAP on the self-made data set, and has better detection effect for small-sized drone targets. Compared with the network before the improvement, the detection speed is increased by 10.3% and the memory cost is saved by 65%, and the requirements of computing capabilities and storage capabilities for devices are reduced, and which is more conducive to practical applications and engineering deployment of drone detection systems.