An integrated interferometric fiber optic gyroscope (I-IFOG) is a promising solution for next-generation interferometric fiber optic gyroscopes (IFOG) owing to its miniaturization and low power consumption. After integrating the optical components onto a chip, a significant polarization error arises because of the absence of polarization-maintaining fiber connections, which significantly reduces the angular velocity measurement accuracy and gyroscope stability. The polarization error greatly restricts further use of the I-IFOG. To analyze the optical path polarization, a polarization error model of the integrated optical chip was established based on a Jones matrix. Using the mathematical model, we have found that the primary factors affecting polarization are the nonreciprocity noise induced by polarization crosstalk and mode dispersion. Therefore, a novel waveguide polarizer is proposed to remove the undesired polarized light and maintain a high degree of polarization. To verify the polarizer performance, the polarization state of the signal was measured using a polarization analyzer. The degree of polarization improved by 7.16% and the extinction ratio improved by 39.8% compared with those of a traditional polarizer. Finally, a prototype based on the referred polarization suppression method was generated to measure the Earth's speed, and the static performance was checked and processed. The accuracy of the prototype reached 0.112 9 (°)/h.