To address the inadequate detection precision and computational efficiency of common traffic sign detection methods under poor lighting conditions, capturing small distant targets, and in complex backgrounds, this study introduces an enhanced YOLOv5s algorithm, named BMGE-YOLOv5s. The proposed method employs BoTNet (bottleneck Transformer network) to replace the original backbone network of YOLOv5s. It also designs a lightweight network, C3GBneckv2, which integrates the GhostNetv2 bottleneck and an efficient channel attention mechanism, reducing the number of parameters while significantly enhancing the feature extraction capability for traffic signs. To further enhance the accuracy of bounding box localization, the MPDIoU loss function is utilized. Experimental results indicate that the improved network model achieves a mean average precision of 93.1% at an intersection ratio threshold of 0.5, indicating an improvement of 3.3 percentage points over the baseline model on the same dataset. Moreover, the proposed model demonstrates a 9.375% decrease in floating-point operations, a ~25.98% decrease in the number of parameters, and a ~67.40% increase in detection speed. The proposed algorithm effectively balances robustness and real-time performance, showing a clear performance advantage over traditional methods.