In order to meet the requirements of high-precision polarization detection, the influence of the polarization effect of the telescope group needs to be considered in the design of the channel-type polarization spectrometer, and the corresponding analysis and optimization should be carried out. First, the influencing factors of the polarization effect of the telescope group are analyzed, and the Mueller matrix model of the telescope group considering the polarization effect of the film system is established by using coordinate transformation and Mueller matrix multiplication method, which is brought into the polarization demodulation model of the channel polarimeter. Next, by simultaneously controlling the transmittance and phase retardation of S light and P light, a corresponding film system with low polarization effect is designed. Finally, the polarization effect simulation of the telescope group coated with different film systems is carried out by using the method of polarization ray tracing. The simulation results show that at the wavelengths of 580 nm and 750 nm, the polarization detection accuracy of the film system with low polarization effect and the film system with high polarization effect does not change significantly. At the wavelength of 420 nm, the polarization detection accuracy of the edge field of view of the low polarization effect film system is 3.22% higher than that of the common film system. The low polarization effect film system effectively reduces the polarization effect of the telescope group of the channel-type polarization spectrometer and improves the polarization detection accuracy of the instrument.