Aiming at the problem that the existing lane detection algorithms are difficult to balance the detection accuracy and speed in practical application, a new lane detection algorithm based on ARM embedded platform is proposed. Firstly, a lightweight semantic segmentation network is designed. When SegNet structure is optimized, skip connections are added to the first layer of the network, and channel attention mechanism modules are added after every two convolutional layers to ensure detection accuracy and improve detection speed. Secondly, Kalman filter lane tracking model is constructed to improve the robustness of detection in video streams. Then, the encoder is reconstructed and the model is lightweight. The deep separable convolution is used instead of the traditional convolution to reduce the calculation cost and improve the detection speed. Finally, the Trt model is generated by TensorRT accelerated reasoning to facilitate its deployment in ARM embedded platform for real-time lane detection. Experimental results on the self-produced Tusimeple extended data set show that the proposed algorithm can cope with various complex traffic scenarios, and its detection accuracy is 98.03%, which is superior to other algorithms. And its detection speed reaches 50 FSP, which meets the real-time detection requirements. This algorithm has high robustness and good real-time performance in complex traffic scenarios, and has certain theoretical and practical value.