In experiment, it has been found that a possible failure in process of solving the fitting coefficients or sudden changes of measurement results can not be absolutely prevented in fitting interference wave surface with Zernike polynomials. The equivalence of the algorithm of least squares method and Gram-Schimdt is strictly proved. It is demonstrated that both two algorithms of solving Zernike polynomial coefficients have the same stability. The basic requirement to ensure the measuring reliability of interference optical detection system is found through a series of comprehensive and systematic experimental researches. The condition is that the order of Zernike polynomial to fit interference wave surface must be less than the number of interference fringes in the diaphragm, and it also has been proved in theory.