In order to solve the problems of low identification efficiency and poor real-time performance to cotton foreign fiber detection, an improved method of cotton foreign fiber detection based on YOLOv3 was proposed. The constructed model of MobileNets-YOLOv3 was designed, in which a lightweight MobileNets network was selected as the feature extraction network, combined with YOLOv3s multi-scale feature fusion detection network. A segmented learning rate was proposed to enhance the learning effect. The cotton foreign fiber image data set collected from the equipment on real site was divided into a training set and a test set, according to a ratio of 4∶1. Six image augmentation methods, such as contrast enhancement, brightness enhancement and so on, was induced, to expand the original data set. Based on the training set before and after expansion, different learning rates, the YOLOv3 model before and after improvement, the proposed model and Faster R-CNN along with SSD_300, different models were tested and compared. The experimental results showed that the data expansion and the segmented learning rate can improve the overfitting of the training model, and the mean average precison (mAP) on the test set was increased by 3.6% and 5.64% respectively. The improved YOLOv3 model can significantly reduce the mistake rate and miss rate of detection. The mAP of the test set was 84.82%, and the frame rate was 66.67 f·s-1. Compared with YOLOv3, the mAP of MobileNets-YOLOv3 was increased by 2.03%, the frame rate was raised to 3 times as it and the duration of training was shortened to 1/4 times as it. The overall performance of MobileNets-YOLOv3 was also better than that of Faster R-CNN and SSD_300. The proposed method can better satisfy the accuracy and real-time performance of cotton foreign fiber detection.