In the phase measuring deflectometry, the calibration accuracy of the system has a decisive influence on the accuracy of the profile measurement. When the calibration target with markers is used for calibration, the error will be introduced because the surface of calibration target is not an ideal plane, which leads to the inaccuracy of solving the pose of the virtual screen image, thus affecting the calibration accuracy. In order to reduce the error, a high precision standard plane mirror is used as a reflector. Starting from the initial system parameters, the system geometric parameters are calibrated using the method of alternating direction optimization, which improves the calibration accuracy and avoids the matrix ill-condition caused by too many variables. In this paper, the binocular phase measurement deflection system with the proposed method is calibrated, and then the standard plane mirror with diameter of 100 mm is measured. The results show that the proposed method improves the accuracy and stability of the calibration compared with the traditional method with a calibration target with markers.