Spectral images, which theoretically have a wider range of applications, store more information than RGB images. However, due to the high cost of spectral imaging equipment and complex data processing, spectral images are mainly applied in remote sensing, military and other fields. In recent years, scholars have proposed solutions to reconstruct spectral images by mathematical methods using RGB images, which can greatly improve the application range of spectral images. However, there are many problems in current spectral reconstruction models, such as the loss of image details and insufficient spectral accuracy. Therefore, this paper proposes a spectral reconstruction method from RGB images based on a dual attention mechanism to improve the quality of spectral image reconstruction from image detail and spectral accuracy. The proposed spectral reconstruction method designs a sparse signal depth reconstruction network, focusing on the sparse characteristics of RGB images, and achieves sparse to complete signals reconstruction by accurately extracting multi-level features of image information and mining more semantic information. Regarding network structure, the designed spectral reconstruction network first uses small parameter convolution to extract shallow feature information of RGB images. Then, the effective multi-frequency channel attention mechanism was used to calculate the correlation between each channel in the feature layer, and the effective distribution of feature response was realized by inter-layer weighting. At the same time, the layer feature weighted fusion attention mechanism is introduced to learn the dependence between features of different layers, and the weights are optimized through different layers weighting to extract effective spectral depth features. Finally, based on the extracted depth features, the hyperspectral image is transformed into a specified dimension by convolution. The experiment uses the python 3.7 programming language, pytorch 1.2, as the deep learning model framework and combined spectral image error and RGB image error as loss functions for the training of the spectral reconstruction network. The proposed method and 7 mainstream spectral reconstruction methods are compared and verified on the NTIRE 2020 and CAVE datasets. From a subjective perspective, the spectral image details recovered by this method are clearer, and the error is smaller. From the perspective of objective indicators, the spectral images reconstructed by this method are reduced by 18.9%, 16.6%, and 22.2% in RRMSE, RSAM and RERGAS indicators, respectively, compared with the methods with better reconstruction performance in the existing literature. The RPSNR indicator improved by 4.5%. Therefore, the experimental results prove the effectiveness of the proposed method from RGB image spectral reconstruction.