To solve the problem of the low accuracy and slow speed of traditional methods for detecting surface defects in steel plates, we propose an improved YOLOv5s algorithm. First, the steel datasets were re-clustered using K-means algorithm based on the intersection-over-union (IoU) metric distance, to obtain multiple groups of anchor boxes; a genetic algorithm was used to perform mutation operations and obtain multiple groups of anchor boxes that match the entire ground truth box better. Second, MixUp was fused with the Mosaic data enhancement to avoid over-fitting and improve the generalizability of the model. Then, the network structure was improved, and an attention module was incorporated to improve feature extraction capability of the network further. Finally, Focal loss was incorporated into the loss function to improve the convergence speed and detection accuracy of the network for hard-to-identify samples. Our experimental results show that the mean average precision (mAP) of the improved YOLOv5s algorithm on a test set is 78.4%, which is 3.0 percentage points higher than that of the original algorithm, and the speed is same as the original YOLOv5s. The detection performance of the improved YOLOv5s algorithm is better than that of DDN, Faster R-CNN, and YOLOv3, and it maintains a high detection speed.