Aiming at the problem of insufficient feature extraction in bridge disease images under complex environmental background and noise, the method of integrating fractal geometric features with YOLOv7 network is proposed to improve the accuracy of disease detection. Firstly, the fractal feature module (FFM) is designed to obtain the fractal feature map of bridge disease images. Secondly, the adaptive feature fusion layer is designed to integrate the extracted fractal features into the YOLOv7 network and the network can obtain more expressive feature map. Finally, the coordinate attention mechanism is introduced to enhance the detection accuracy of the network for small diseases. The experiment examines the complex images of five bridge diseases including efflorescence, crack, exposedbars, corrosionstain and spallation. The results show that, with the same dataset and numbers of iteration, the mean average precision of YOLOv7 network increases from 82.94% to 86.24%, and the average accuracy of crack disease detection increases the most significantly, from 75.92% to 81.29%.