At present, most of the research on forestry pest detection is based on traditional machine learning algorithms, and there are some problems such as low accuracy and poor effect. Therefore, a new forestry pest detection model based on YOLOv4, Pest-YOLOv4, is proposed. K-means++ algorithm is used to cluster anchor boxes to obtain a higher avg-IoU value. Combining ECA(Efficient Channel Attention) and CBAM(Convolutional Block Attention Module) to form ECA-CBAM attention mechanism, the network can pay more attention to the characteristic information. Reorganizing the network neck to SPP-PANet, the feature information can be fused with multiple receptive fields effectively. Using Focal Loss to improve the loss function, the learning of samples that are difficult to distinguish is paid attention, and the proportion of positive samples and negative samples are balanced. The experimental results show that Pest-YOLOv4 mAP reaches 90.4%, 4.2% higher than YOLOv4, while the FPS remains at 33.4 f/s, which meets the detection accuracy and real-time requirement of forestry pest detection tasks.