In the aerial image of unmanned aerial vehicle(UAV), the target is usually small, and the shooting angle and height are variable. To address the problems, we proposed an adaptive drone object detection algorithm based on the multi-scale feature fusion. First, lightweight feature extraction network was established using the advantages of deep separable convolution and residual learning. Second, a multi-scale adaptive candidate region generation network was constructed, and feature maps with the same spatial size were weighted and merged based on the channel dimensions, which enhance the feature expression ability to objects. Based on these multi-scale featured maps, the use of semantic features to generate target candidate frames can be more matchable with real objects. Moreover, simulation experiments demonstrate that this algorithm can effectively improve the accuracy of UAV detection and have better robustness.