Extracting the line drawings of ancient frescoes using existing edge detection methods suffers from high noise interference and more information loss. In this paper, we propose a fusion pixel difference convolution method to extract the optimal band of mural lines. The minimum noise separation method is used to separate the effective information and noise from the multispectral data of the mural, and the optimal principal component band is selected for the extraction of the line art. For the problem of traditional convolution to extract image gradient information, pixel difference convolution is introduced to improve the image gradient information for edge detection. A scale enhancement module (SEM) is added to the side output network to enrich the multiscale features. Meanwhile, for the pixel misclassification issues caused by pixel level imbalance, Dice loss function strategy based on image similarity is designed to minimize the pixel distance step by step to obtain clear image edges, and the mural dataset prior knowledge fine-tuning model is used to solve the problem of insufficient dataset. The experimental results show that the method in this paper can extract clearer line drawings in scenes with faded and noisy murals, and the SSIM and RMSE of the line drawing images are better than other algorithms, improving 2%~10% and 2%~4%, respectively, compared with PiDiNet. The model is validated on the public dataset BIPED, and the ODS and OIS of the proposed method are improved compared with PiDiNet by 0.005 and 0.007, respectively. The method can extract clear and complete line images for faded and diseased murals.