A new method for improving accuracy of real-time three-dimensional (3D) measurement based on a composite grating is proposed. By analyzing the composite phase measuring profilometry (PMP), it is found that phase demodulation precision is related to filtering window choice when demodulating the deformed phase shift stripes from a deformed composite grating. By analyzing and comparing several common filtering windows, a new kind of filtering window which is combined Hanning and rectangular window together is designed. Because filtering precision is related to distribution of spectrum, by digital simulating Peaks-function object in 3D measurement based on a composite grating, distribution of optimized parameters of combined window is obtained. Using these data, the proper object in experiment to improve the accuracy of real-time 3D measurement based on a composite grating can be realized. Both simulating and actual experimental results prove the method′s effectiveness and practicability.