Road damage detection is of vital importance in the process of road maintenance. In order to solve the problem that the current road damage detection relies on large vehicle-mounted detection equipment, resulting in high cost and poor universality, an ultra-lightweight surface crack detection model applied to edge equipment was proposed. Firstly, a lightweight road damage detection network was constructed, and on this basis, redundant parameters in the network were deleted based on the feature mapping contribution pruning strategy, so as to ensure real-time monitoring in edge devices without reducing the detection accuracy. Finally, the experimental results show that the size of the proposed model is only 2.06 M, and the accuracy rate reaches 68.6% at the frame rate of 40. Compared with the original YOLOv5s model, when the accuracy is only reduced by 3.4%, the number of parameters is reduced by 70.8%, and the inference delay on the edge platform is reduced by 175 ms. In summary, the proposed model can detect road characteristics more effectively while maintaining a high level of accuracy, and has good robustness for road damage detection.