Filtering is one of the key issues in interferogram processing. In order to solve the long running time of windowed Fourier filtering in processing of interference pattern, a fast Fourier transform in windowed Fourier filtering is adopted to reduce the computation time and overcome the faults of local fringe information cannot be extracted by Fourier transform. A fast windowed Fourier filtering method has been proposed, which can reduce the computation time significantly comparing with convolution method of windowed Fourier filtering without sacrificing measurement accuracy. It has solved the long time-consuming fundamentally and made windowed Fourier filtering method be suitable for real-time speckle detecting. Experiments verify this method is fast, effective and reliable.