A concrete crack detection method based on improved YOLOv4 is proposed to address the problems of low detection accuracy, large number of model parameters and slow detection speed in current deep learning methods for detecting concrete cracks. Firstly, the backbone feature extraction network of YOLOv4 is replaced by the lightweight network Mobilenetv1, and the ordinary standard convolution in the enhanced feature extraction network of YOLOv4 is modified into a depth-separable convolution; secondly, the lightweight attention module CBAM (Convolutional Block Attention Module) in the PANet module to improve the accuracy of crack target detection with a controlled amount of parameters; finally, the Spatial Pyramid Pooling (SPP) module in YOLOv4 is replaced by the RFB-s module that simulates human vision. The experimental results show that compared to conventional YOLOv4, the mAP of this model increases by three percentage points, the amount of parameters is reduced to 14 M and the detection speed is up to 42 frames per second.