Pigments not only endow cultural relics with color and aesthetic value but also carry rich historical, cultural, and technical information. Accurate classification and identification of pigments are essential for the restoration, preservation, and academic study of ancient painted artworks. Identifying the types and compositions of pigments helps determine the creation period, regional characteristics, and craftsmanship style, providing scientific guidance for restoration and cultural value research. However, traditional pigment analysis faces challenges due to limitations in sample size, surface flatness, and the destructive nature of some analytical methods, which may cause irreversible damage to the artifacts. Hyperspectral Imaging (HSI), with its non-destructive nature, wide-area scanning, and capability of capturing complete spectral information, has become a powerful tool for pigment detection. HSI overcomes the limitations imposed by uneven surfaces and small sample sizes, enabling the extraction of fine-grained spectral and spatial features from pigments. This study aims to utilize HSI for the precise classification and detailed feature extraction of ancient painting pigments, addressing challenges in complex scenarios. We propose a multi-scale spatial-spectral feature fusion method to integrate information at different levels. A spectral-spatial attention mechanism is employed to capture fine details. At the same time, the Vision Transformer (ViT) extracts high-level semantic information from the entire image, enhancing the representation of complex pigment features and improving classification performance. The experimental results show that the proposed method significantly outperforms traditional and other deep learning models in the classification of simulated painting samples: it improves classification accuracy by 34.35% compared to the Support Vector Machine (SVM) and by 8.93% and 5.6% compared to HyBridSN and Spectral-Spatial Residual Network (SSRN), respectively. This study not only improves the accuracy of pigment detection but also provides non-destructive, reliable technical support for the scientific restoration and cultural value preservation of ancient paintings, contributing to the intelligent development of cultural heritage conservation.