In order to improve the recognition accuracy and implementation performance of road traffic signs, an improved LeNet-5 convolutional neural network structure is proposed to train 18 kinds of traffic sign images. Firstly, in the detection phase, the light-weight segmentation algorithm based on color and Hough transform algorithm is used to extract the target area of traffic signs, and the complexity of the control algorithm makes the recognition system basically meet the real-time requirements, and then LeNet-5 is used to classify and recognize traffic signs. The results show that the traffic signs are successfully identified by driving in the algorithm of this paper, and the running speed reaches 16.9 Hz, which basically meets the performance requirements of traffic sign recognition, such as stability, real-time and so on.