In the field of optical measurement, interferometry has extremely important significance. For phase extraction of fringe pattern, a new algorithm based on two-dimensional continuous wavelet is proposed. For reducing calculation time, the fast Fourier transform is adopted to the process and discrete scale factors and rotation angles are chosen instead of continuous scale factors and rotation angles used in traditional two-dimensional continuous wavelet transform. So the long calculation time issue for two-dimensional continuous wavelet transform algorithm is resolved. Simulations and experiments show that the phases of fringe patterns can be extracted fast and accurately with the algorithm. For the fringe patterns with defects, image inpainting can be realized and the phase patterns can be obtained. And a good effect is obtained.