Spectral holography, as a holographic imaging technology, can simultaneously record the spatial and spectral information of objects and holds promising application prospects. In the optical path of spectral holography recording, a mirror driven by a high-precision displacement stage is utilized to control the phase-shift of the reference beam and to record the corresponding white-light hologram. This is a crucial step in achieving the reconstruction of monochrome holograms, and subsequently the reconstruction of an object's spatial and spectral information. To realize high-quality reconstruction of the original information, the effects of different displacement parameters of the moving mirror on the reconstructed image quality are thoroughly analyzed. Firstly, the impact of the displacement step number on the reconstructed image quality is studied, and the optimal displacement step number under different reconstruction wavelength conditions is determined. Under the condition of the above displacement step, the influence of displacement symmetry on the reconstructed image quality is analyzed, and a parameter for displacement asymmetry is proposed, along with its relationship to reconstruction quality. Finally, the influence of displacement accuracy on the reconstructed image quality under different reconstruction wavelength conditions is analyzed under the optimal conditions of the aforementioned two parameters. The results show that considering the holographic recording speed, the amount of recorded hologram data, and the reconstructed image quality, the larger the number of displacement steps, the smaller the asymmetry, the higher the displacement accuracy and the higher the reconstructed image quality. These findings provide robust supports for the selection of electronically controlled displacement stages and the precise control of displacement parameters in experiments.