A method by using magneto-optical modulation and light source modulation techniques was proposed to measure the extinction ratio of a polarizing prism precisely, and an extinction ratio measurement system was also established based on the two modulation techniques. Firstly, the system′s measurement model was derived according to the Jones matrix describing manner of polarized light. By using a magneto-optical modulator, the optical axis angle between a polarizer and a measured polarizing prism was precisely located. Then, by using a chopper to modulate the square wave of a light source, the impact of noises on the system measurement accuracy was eliminated and the system could measure the light intensity value accurately after the measured polarizing prism′s axis was located precisely. Finally, the polarizing prism was measured for a number of times and the data were averaged. The experimental results indicate that the measurement accuracy of extinction ratio for the polarizing prism can be 10-6, which verifies the effectiveness and stability of the method. The system is characterized by high stability, high accuracy, and can offer the references for the performance test and engineering applications of polarized devices.