For the problem of being susceptible to the influence of noise, low accuracy in micro in-plane displacement measurement using traditional digital speckle correlation in frequency domain, digital higher-order vortex speckle correlation method in frequency domain is proposed. First, it is found that higher-order vortex speckle pattern has an ability to resist noise and the output surface of correlator can get smaller and brighter correlation points according to the experiment of generating higher-order vortex speckle pattern, then using higher-order speckle pattern as measured object. Second, theoretical calculations are deduced, and then new patterns before and after displacement of the higher-order vortex speckle are synthesized using subtract operations. The two correlation points can be gotten on the output of correlator, which changes the current status that only has one correlation point using traditional method, effectively counteracting background noise. Finally, we design a new filter to generate the relevant spot diagram, whose correlation spot map peaks become sharp. Experiment results show that the algorithm has made great advance in reducing noise and improving accuracy, which can maintain relative error less than 2% under different noise.vortex; matching filter