Spectral imaging can record the spectral information of the scene to achieve high-fidelity color reproduction. The spectral image is usually mixed with the spectral information of the object and the light source information at the same time, so the existing color reproduction methods of spectral imaging need to obtain the light source information of the environment through the calibration target placed in advance or the object with known spectral characteristics, and then correct the color constancy of the image. However, it is usually difficult to meet the above conditions in the actual use of spectral cameras, which brings challenges for image high-fidelity color reproduction. A calculation method of image color constancy for spectral imaging was proposed. The spectral data obtained by the spectral imaging system were converted into the XYZ color space, and the image was divided into regions for statistics to get the statistical points of the image. According to the distribution rule of many common light sources, position weights and color temperature weights were applied to statistical points, and luminance weights were set to remove over-dark and over-saturated statistical points in the image. The chromaticity parameters of ambient light were obtained by weighted average. Moreover, the XYZ color space data of the image were converted to RGB color space according to the application requirements. The gains of different channels of the image were calculated according to the chromaticity parameters of ambient light to complete the spectral image’s color constancy calculation. In order to verify the effectiveness of the proposed algorithm, 140 spectral images were processed, and the reproduction angle error between the chromaticity of the ambient light obtained by the algorithm and the chromaticity of the real light source was calculated. The correction results showed that the proposed algorithm was better than the spectral images without processing and the Gray-world method. In order to further analyze the relationship between the algorithm correction results and human perception, a color psychophysics experiment was designed, 18 observers with normal vision participated in the experiment. The average score of all observers for the algorithm correction results was between good and excellent, which can meet the actual use needs. Along the direction of the average color temperature line, the chromaticity difference of the light source accepted by the observer was slightly larger than in other directions. When the image contained a large area of memory color, and the chromaticity of the light source shifted in a direction that increased the saturation of the memory color, the chromaticity difference accepted by the observer became larger. The results of objective and psychophysical experiments show that the proposed algorithm can deal with the image color constancy in spectral imaging well when the light source is unknown, and there is no calibration target, which lays a foundation for high-fidelity color reproduction of spectral images.