Generally, deep learning methods are used to identify and classify the foreign fibers in cotton. First, a target recognition framework is adopted based on Faster RCNN to develop a foreign fiber dataset according to the characteristics of foreign fiber size and shape diversity. Next, the original VGG16 is replaced by RseNet-50 as the feature extraction network in the foreign fiber classification model, and the size of the mark box is improved using the k-means++ clustering algorithm. Subsequently, the model is trained to identify and classify the foreign fibers in cotton. The trained model achieves an accuracy rate of 94.24%, a precision of 98.16%, a recall rate of 95.93%, and an F₁ score of 0.970 with respect to the verification set. When compared with the original model, the recognition effect is observed to improve in case of small sizes, large aspect ratios, and dense occurrences when the proposed model is used. Furthermore, the accuracy, precision, recall rate, and F₁ score of the proposed model improve by 3.21%, 0.90%, 2.51%, and 0.017, respectively, when compared with those of the original model.