The influence and origin of quadrature phase shift on the measured rectilinear displacement measurement error was investigated using a homodyne laser interferometer with a quadrature output. Common nonlinearities, including zero-shift, different amplifier gains and phase shift are three typical errors for displacement measurements. A simple method was used to determine and correct the zero shift and unequal gains, and the influence of the phase-shift was described in detailed experimentally, including the effective way to correct phase error. The effectiveness of the method was validated by comparing the displacement error before and after phase shift error-corrected. Moreover, the method can be applied to assess standard machine and adjust interferometer.