To tackle the challenges of reliance on manual inspection, low detection efficiency, and high costs in detecting surface defects on worm gear teeth, automated methods utilizing machine vision were researched. A defect collection system was designed to capture worm gear defects, and data augmentation strategies were introduced to handle varying defect occurrence rates. Enhancements were made to the YOLOv7 algorithm. Firstly, to address the differences in defect size distribution, a progressive feature pyramid was incorporated to reconstruct the neck network, improving the model's multi-scale feature fusion capability. Secondly, an attention mechanism was added to minimize interference from non-defective areas and bolster the model's focus on defects. Lastly, the regression loss function was modified to SIOU, and orientation consideration was included during network training to boost detection accuracy. Ablation experiments demonstrated the effectiveness of these improvements. With a 20.7% reduction in parameter count, experimental results show that the proposed algorithm achieves a 3.3 percentage point increase in accuracy compared to the YOLOv7 algorithm. Additionally, when compared to other algorithms like YOLOR and YOLOv5m, this algorithm provides optimal detection performance, effectively meeting the requirement for automated detection of surface defects in worm gears.