Aiming at the problems of low accuracy and low efficiency of wheel set tread defect detection of high-speed trains, an improved YOLOv5 algorithm is proposed to realize fast and accurate detection. A convolution attention mechanism is introduced to optimize the features in the channel and spatial dimensions so that the essential target features occupy a greater proportion in the network processing to enhance feature learning ability in the target region. According to the size of the tread defect category, the structure of the Neck area is simplified, and the characteristic graph branches suitable for detecting small- and medium-sized targets are retained to reduce the model's complexity. The loss function of the bounding box regression is changed to efficient intersection over union (EIoU) to integrate more bounding box information and improve prediction accuracy. Compared with the original YOLOv5 algorithm, experimental results demonstrate that the mean average precision (mAP) of the enhanced YOLOv5 algorithm on the test set is increased by 5.5 percentage points, and the detection speed is improved by 2.8 frames/s, which has a strong generalization ability in complex scenes.