In order to solve the drawbacks of the prism spectral imaging system, such as poor direct vision, low spectral linearity and large natural smile, a spectral imaging system based on double Amici prism is proposed. MATLAB software is used to realize the accurate calculation of the prism smile based on vector refraction law, which has been verified by simulation.On the basis of studying smile characteristics of double Amici prism, and as the dispersive element, a direct vision double Amici prism with smile eliminated is built. The wave band of the designed system is 400nm to 800nm, and the spectral resolution in the full waveband is higher than 7nm. At the central wavelength of 600nm, the angle between the chief ray of the center view refracted by double Amici prism and the optical axis is less than 0.31°. Moreover, the maximum value of the smile in the full waveband is less than 2.557μm, which can meet the requirements of direct vision and eliminating smile. It’s of general guiding significance to the design of prism spectral imaging system with smile eliminated.