In the process of the parameter calibration of projector camera system, the recognition accuracy of corner is low and the noise immunity is poor. To solve the above problems, a new projection color pattern feature image and a sub-pixel corner detection algorithm are introduced to increase the accuracy of corner detection and recognition. In the process of photon signal transmission, the projector-camera can cause photon signal loss due to the coupling of system channel, therefore, the system coupling is modeled and analyzed. And a system coupling correction scheme is proposed, which effectively reduces the system coupling crosstalk. In the process of calculating the system calibration parameters, due to the interference of external factors, the error of the calibration parameters of the projector is large. Considering that the projector camera has epipolar geometry constraint relationship, a projective geometry constrained optimization method based on imaging feedback is proposed to optimize the system parameters. The experimental results show that this method can achieve 0.25 pixel accuracy. Meanwhile, it has high plane parallelism and linear verticality. Projection distortion correction experiments show that the distortion correction effect of projection screen basically accords with human visual viewing consistency.