Aiming at the fact that the existing infrared polarization image pseudo color technology can only display the collected polarization image scalar and non dynamically, the infrared pseudo color signal processing based on P and S polarization image is studied, and the P and S polarization image information is mapped to RGB color space after processing. The hardware circuit of pseudo color processing system with TMS320DM642 chip as the core processor is designed, and the software design, debugging and analysis of pseudo color signal processing system are completed. By collecting the infrared polarization characteristics reflected from the surface of the measured object, the infrared polarization image processing system obtains more useful information and ensures the processing speed of the infrared polarization image. It is of great significance for infrared target recognition and detection.