A nonlinearity correction method for a homodyne polarized laser interferometer is proposed, which can compensate nonlinearity error for the interferometer in realtime. Because of the parameter errors, adjusting errors of optical components environment disturbing effect, the output signals′ orthogonal characteristic is destroyed, which causes the nonlinearity error. Based on the ellipse fitting principle, nonlinearity error correction method is proposed. It doses a realtime estimation of the interference signal′s parameter characteristics and studies phase correction range that affects the correction error, then fractionizes signal by 16384 subdivisions, then a result with picometer resolution is realized. A correction criterion for realtime nonlinearity correction is proposed. A homodyne polarized laser interferometer and vibration measurement facility are established. The experimental results indicate that the signalnoise ratio of the interferometer with the nonlinear correction is 30 dB more than the one and measurement resolution is superior to 10 pm/Hz^1/2.