For handling low detection accuracy, poor real-time performance, and large model size of small target traffic sign detection, a real-time road traffic sign detection algorithm based on Yolov5 is proposed. First, the inverted residual structure in Mobilenetv3 was improved and applied to the backbone network of Yolov5 to align with the lightweight network design. Then, the lightweight upsampling universal operator CARAFE (content-aware ReAssembly of FEatures) replaced the nearest neighbor interpolation upsampling module of the original network, reducing the loss of upsampling information and increasing the receptive field. Finally, global and local fusion attention (GLFA) was used to focus on the global and local scales to enhance the sensitivity of the network for small target objects. Experiments on the self-made Chinese multiclass traffic sign dataset (CMTSD) show that the enhanced algorithm improves mean accuracy precision (mAP) @0.5 by 2.58 percentage points based on the model size reduction of 8.76 MB compared with the algorithm before enhancement. Furthermore, the detection speed reaches 62.59 frame/s. Compared with other mainstream object detection algorithms, the proposed algorithm exhibits certain advantages in detection accuracy, speed, and model volume and performs better in real complex traffic scenes.