To alleviate the effect of local highlight and self occlusion of a random-pattern-projection-based binocular stereo measurement system, a novel method by using a single camera and the projector as a monocular measurement unit was presented. Firstly, a simple and effective calibration algorithm was proposed to construct the line functions of the rays emitted from every point of the random pattern plate through a projector lens,then the calibration was carried out by just taking several images (called calibrated images) of a white board with random illumination in different orientations. It is unnecessary to calibrate the distortion of projector lens, and the system precision is not affected by the distortion. Then, the object image captured in random illumination was matched with the calibrated image at sub-pixel location, and the 3D point cloud data of the object were obtained by triangulation principle. Finally, the standard plane and cylinder were measured to verify the accuracy of the proposed method, which demonstrates that the measurement uncertainty is less than 0.05 mm (3σ). Several other measurement examples indicate that two monocular measurement units working together can effectively avoid point losing caused by the highlight and occlusion.