The environment at cultural relics protection sites restricts the ability to image large-area painted cultural relics at once, necessitating the use of multi-lens imaging to acquire complete high spatial resolution multispectral data. However, challenges such as uneven illumination, spectral noise, and other disturbances during split-lens imaging can cause spectral dimension offsets, reducing the accuracy of pigment classification. To address this issue, a method for classifying pigments in painted cultural relics using cross-scene multispectral imaging resistant to spectral disturbances has been proposed. First, the overlapping regions of adjacent sub-shot images are extracted based on scale-invariant features, and the histogram specification method is used to eliminate the spectral shifts with the mean gray value of the overlapping regions as the benchmark. Spatial-spectral features are extracted through a deep codec, which randomizes the generation of variable spatial-spectral information to endow it with cross-scene domain shift properties. The model's responsiveness to key spectral channels is enhanced through the spectral channel attention mechanism. Optimization of the generator via an adversarial learning strategy further enhances model generalization capability. The experimental results from simulated and real mural painting datasets demonstrate that the algorithm, utilizing anti-spectral perturbation data, achieves an average improvement of 4.13% in overall classification accuracy and a 5.65% increase in the Kappa coefficient. In cross-scene painted cultural relics pigment classification experiments, the overall classification accuracy of the pigment is enhanced by 4.01%, and the Kappa coefficient by 3.16%.