Defect detection is a challenging problem due to the diversity of appearances, sizes and locations of button surface defects. A low-rank information based button image reconstruction method is proposed based on the spatial structure correlation of defect image information, in which the low-rank constrained defect image matrix is utilized to reconstruct the defect-free button surface images through regression and the background subtraction method is adopted to separate the residual images with defect information, and thus the defects can be effectively extracted through the locally weighted adaptive threshold. In addition, in this method, the minimum rank of the residual matrix is converted into the minimum nuclear norm, the regression coefficients are solved by the alternating direction multiplier method, and thus the image reconstruction is realized with positive samples. According to the performance test of the reconstructed button sample set, it is verified that the proposed method is effective for different types of buttons and different sizes and shapes of defects, and the accuracy of the algorithm is 99%. Moreover, the method has a certain adaptability to illumination variation.