In order to improve the recognition rate of the traditional local binary pattern (LBP) algorithm when extracting target image features, a feature extraction method based on mask iterative region of interest (ROI) to improve the LBP algorithm is proposed. The extraction method using mask iterative ROI reduces the processing of interference information or invalid regions and shortens the extraction time of defective region. Based on the LBP, the circular area of the said central pixel point is determined according to the preset radius, the gray value size relationship between the neighboring sampling points is added into the consideration, and together with the central threshold, it is used as the influence factor to decide the LBP coding situation, and the directional features hidden between the neighboring points are fully utilized to further improve the accuracy of image recognition, and the experimental results show that using the PASCAL VOC gear defect dataset as the validation sample, the defect images captured in the experiment show a 2% improvement with SVM recognition accuracy compared with traditional LBP algorithm, with a maximum recognition rate of 99.32%. The Manhattan recognition accuracy improves by 0.67% compared with traditional LBP algorithm, with a maximum recognition rate of 98.54%. The European recognition accuracy improves by 0.44% compared with traditional LBP algorithm, with a maximum recognition rate of 97.87%.