In optical linear frequency modulated continuous wave (LFMCW) lidar system, due to the limit of semiconductor laser performance, the ideal linear frequency modulation (LFM) signal cannot be obtained under a LFM electrical signal modulation in application, which makes beat frequency signal (BFS) distorted. Thus it is necessary to study the frequency modulated (FM) nonlinear response characteristics. The FM characteristic of semiconductor laser is analyzed and the FM nonlinear response formulas of lidar are derived. Discrete polynomial phase transform (DPT) algorithm is applied to estimate and compensate on the FM nonlinear parameters. And experimental platform is set up for validation. The experimental results show that the DPT algorithm can effectively estimate and compensate lidar FM nonlinear parameters and improve the accuracy of velocity and range.