To solve the problem of traditional surface defect detection's inadaptability to a complex industrial background, a surface-defect detection algorithm based on feature pyramid matching and self-supervision is proposed. First, the features extracted from two ResNet networks based on channel attention are structured into a pyramid, which enables defect detection using the output differences of each layer of the network. Second, bootstrap your own latent (BYOL) self-supervised learning is used in the training mode of the pretrained network; the network with self-supervised learning can extract general features and improve the generalization of the defect detection method. Finally, for fuzzy images, distillation training based on different resolutions is used to let a student network fully learn to extract the depth features of images. The proposed algorithm is tested on three datasets. Experiments show that the proposed method is better than the control group and has a higher defect detection accuracy.