After analyzing the traditional calibration model for target deviations of photoelectric theodolites and the characteristics of photoelectric theodolites with multi-field of view stitching, we derive a calibration formula for target deviations of photoelectric theodolites with imaging systems that have large collimation errors and zero offsets according to the principle of coordinate transformation. The above calibration formula and target simulator pointing are used to reversely deduce the calculation formula of target deviations of photoelectric theodolites with large collimation errors and zero offsets. The pointing calibration coefficient of the imaging system is solved through its actual target deviation. A verification test shows that the proposed approach breaks through the limitations of the existing distortion correction model and can be applied to pointing calibration of the imaging systems of photoelectric theodolites with multi-field of view stitching. The measurement system with a 2×3 externally stitched array discussed in this paper has a collimation error of 11.26° and a zero offset of 18.08°. Both the horizontal and vertical pointing errors are less than 1/5 pixel after the system is calibrated by the pointing calibration method for photoelectric theodolites with multiple externally stitched imaging modules.