To determine the influence of carrier phase delay and modulation depth drift on the accuracy of phase demodulation in sinusoidal phase modulation interferometers, a phase-generated carrier (PGC) demodulation nonlinear error compensation method based on a Kalman filter is proposed. First, we establish a Kalman filter state space observation model composed of PGC orthogonal component parameters. Next, the amplitude and offset of the orthogonal components of the PGC demodulation are optimally estimated and corrected to reduce the nonlinear error of the phase demodulation. Finally, a theoretical analysis is conducted and used to explain the principle of the compensation method, and we carry out the phase demodulation simulation test of an analog interference signal and a sinusoidal phase modulation interference displacement measurement experiment. The experimental results show that the proposed method can effectively reduce the nonlinear error of PGC phase demodulation and facilitate displacement measurement with nanometer precision.