In order to improve the measuring accuracy of double fibers point diffraction interferometer, system errors of two wavefront reference sources (WRS) should be calibrated. A calibrating process and calibrating algorithm of WRS are designed. The non-axisymmetric errors of WRS about optical axis are obtained through rotating the tested optical system around optical axis by four times. The non-axisymmetric errors of WRS about oblique axis are obtained through rotating the tested optical system around oblique axis by four times. Then the axisymmetric errors of WRS are gotten by the least squares, and the system errors of WRS are equal to the sum of the axisymmetric errors and non-axisymmetric errors. The process of calibrating algorithm is simulated by computer. When the rotating tolerances of WRS are in the range of ±1°, the root mean square value of measuring error is less than 0.01 nm using the 36 terms of Zernike polynomials, which is in the range of calibrating accuracy. The validity of calibrating algorithm is verified and the rotating tolerances of WRS are given by simulating the calibration.