An on-line three-dimensional (3D) measuring method based on oriented FAST and rotated BRIEF (ORB) algorithm is proposed to match feature points of moving objects. Only a fixed sinusoidal grating is projected onto the measured object with uniform motion, and the CCD is used to take photos at equal time intervals to obtain the five deformed patterns on the object and extract the background patterns respectively. With the aid of the ORB feature points matching algorithm on the background patterns, the displacement of the object in the last four frames relative to the first frame can be obtained. So a group of deformed patterns with the same shifting phase can be cut out. And then the 3D surface shape of the object can be reconstructed successfully with an equal phase-shifting algorithm. The experimental results show the feasibility of the proposed method. And it is suitable for on-line 3D measurement.