In order to improve the quality of image scratch detection, adaptive binary tree algorithm is proposed. Scratch area is determined by projection of image, image sharpening is operated on mask. Gray of image scratch in neighborhood is divided using cluster method, the result is used as input node of binary tree, image pixel value is divided into eight stages of binary tree, characteristics of binary tree traverse of the feature pixel, adaptive threshold merge the region of the scratch in neighborthood. Algorithmic process is described. Experimental simulation shows the proposed algorithm can accurately detect the location and direction of the scratch, and is good for qualitative detection index.