In order to solve the problems of low efficiency, high false detection rate, low versatility, poor real-time performance of the current fruit identification and detection methods, a fruit recognition detection method based on improved you only look once (YOLO) YOLOv4 algorithm was proposed in this study. Firstly, an efficient channel attention was added to the backbone network to enhance the network’s ability to extract semantic information from images. Secondly, the convolutional layers at the cross stage partial block junction in the backbone network were replaced by involutions, which reduced the model size and enhanced the network prediction performance. Finally, residual modules were added to the feature fusion network path aggregation network to speed up network convergence and prevent network gradient explosion. The datasets selected 10 kinds of fruits common in life: dragon fruits, oranges, grapes, green mangoes and so on with a total of 6670 pictures. The experiments show that the mean average precision (MAP) of the proposed method is 99.10%, the precision is 95.62%, and the number of frames transmitted is 41.67/s, respectively. MAP is improved by 15.3% compared with YOLOv4. This study meets the requirements of high detection accuracy and detection speed and has important reference value for improving the accuracy of fruit identification.