In a complex construction environment, the helmet wearing recognition algorithm based on machine vision technology often fails and misdetects, and its recognition ability is limited. In order to improve the accuracy of helmet wearing recognition, this paper establishes a helmet recognition convolutional neural network based on the bidirectional feature pyramid of the attention mechanism isproposed. In order to improve the expression ability of shallow position information and deep semantic information in the convolutional neural network, and increase the recognition rate of vague and small helmets, the network introduces the jump connection and the attention mechanism CBAM technology into the bidirectional feature fusion feature pyramid network PANet module, and bidirectional feature pyramid module based on the attention mechanism is constructed. In order to improve the convergence ability of the model, CIoU is used instead of IoU to optimize the anchor frame regression prediction, which reduces the complexity of the network training. The results of comparative experiment show that the mAP value of our proposed recognition network is 0.82, 4.43, 23.12 and 23.96 higher than that of YOLOv3, RFBNet, SSD, Faster RCNN, respectively, and its detection speed reaches 21 frame/s, thus satisfy with height real-time accuracy of helmet recognition in the construction environment.