The generation of different modulation formats and their polarization dispersion compensation (PMD) performance have been studied. Firstly, the eye diagrams and spectrum of different modulation formats through experiment have been obtained, and then the dynamic combination of their mitigation and compensation in PMD have been accomplished. Secondly, the PMD compensation performance with degree of bolarization (DOP) as feedback signal of the mentioned modulation formats has been studied via simulation. It shows that DOP responsibility and PMD compensation performance of differential phase-shift key (DPSK) are better than that of on-off keying (OOK). CSRZDPSK combines the carrier-suppressed characteristic of CSRZ formats and specific spectrum characteristic of DPSK formats, therefore it has the best PMD compensation performance.