BGO crystals usually need to be cut, grind and other rough machining steps to make the grinding surface to improve the performance of components. The extraction and pre-inspection of scratch defects in the rough machining process are very important for the quality evaluation of subsequent crystal components. However, the traditional industrial machine vision algorithm is difficult to finely segment the weak scratches on the rough grinding surface of crystal, which greatly affects the detection efficiency of the subsequent crystal quality. To address the issue of accurately segmenting weak scratches on the crystal grinding surface, this paper adopts an improved U-Net deep learning algorithm. The algorithm embeds a lightweight CBAM attention mechanism into the U-Net architecture to enhance the network’s ability to extract shallow scratch features and recover details. Meanwhile, the Copy-paste data augmentation method is employed to improve the generalization of the algorithm model. In addition, in order to alleviate the negative impact of foreground background imbalance in the sample, the loss function adopts Dice Loss and Focal Loss composite multi-loss function. Experimental results show that the proposed algorithm effectively and accurately segments the weak scratches on the rough grinding surface of the crystal, achieving Miou value of 85.2% and accuracy value of 95.4%, which represents an improvement over traditional industrial machine vision algorithms. Furthermore, the algorithm alleviates the issues of false segmentation and under-segmentation of weak scratches to some extent, enabling the pre-detection of scratch defects in the rough machining process, and ultimately reducing unnecessary processes and quality assessment steps in the future, while overall improving the production efficiency of industrial crystal products.