Small object detection in UAV aerial images faces challenges such as small object sizes, complex backgrounds, and limited computational resources. Most existing object detection models deployed on UAVs suffer from low accuracy and struggle to achieve a good balance between detection accuracy and efficiency. To address these issues, this paper proposes a lightweight small object detection algorithm, ACFI-YOLO11 (Attention-based Cross-layer Feature Interaction-YOLO11), based on YOLO11 framework. First, this paper designs a Tiny Head branch, which enhances the model’s ability to detect tiny objects by introducing higher-resolution feature maps. Second, this paper proposes a novel attention-based cross-layer feature interaction module (ACFI). This module uses a layer feature aggregation (LFA) mechanism and a Transformer encoder to enable direct information exchange between the current layer and its adjacent layers. This approach addresses the limitations of the original model’s neck network, where features were passed sequentially and primarily focused on the preceding layer, failing to fully explore and leverage cross-layer feature correlations. The proposed module significantly enhances the model’s representational capacity. Finally, this paper introduces space-to-depth (SPD) convolution to replace traditional convolution. This reduces the model’s parameters and computational cost while preserving critical spatial information during downsampling, thereby improving detection accuracy for small objects. Experimental results on the VisDrone2021 dataset show that compared to YOLO11s, ACFI-YOLO11 achieves improvements of 4.2%, 3.5%, 5.2%, and 4.0% in AP_S, AP_XS, mAP₅₀, and mAP_50-95, respectively, and outperforms other comparison algorithms with a mAP_50-95 of 31.7%. Furthermore, comparative experiments on the UAVDT dataset validate the superiority of ACFI-YOLO11, achieving a mAP_50-95 of 83.3%, significantly outperforming other state-of-the-art algorithms. These results demonstrate that ACFI-YOLO11 not only achieves a lightweight model design but also significantly enhances detection performance, providing an efficient and practical solution for small object detection in drone aerial imagery.