To solve the problem of low segmentation accuracy of metal workpiece surface defects, we propose a workpiece surface defect segmentation model based on a U-net network combined with a multi-scale adaptive-pattern feature extraction and bottleneck attention module. First, we embed a multi-feature attention aggregation module in the network to improve the utilization of information and extract more relevant features, so as to extract defect targets with high accuracy. Then, the bottleneck attention modules are introduced into the network to increase the weight of defect targets, optimize the extraction of features, and obtain more feature information, thus obtaining better segmentation accuracy. The improved network mean pixel accuracy reaches 0.8749, which is 2.92% higher than the original network. The mean intersection over union reaches 0.8625, an increase of 3.72%. Compared to the original network, the improved network has better segmentation accuracy and segmentation results.