An accurate calibration method for stereo microscope measurement systems was proposed based on a projection model, a non-parametric distortion model and the bundle adjustment algorithm. The key technologies involved were investigated. Firstly, a calibration target was designed by using the lithography method, and the images of the target were recorded from different orientations by using two cameras. Then, the distortion correction fields were calculated by using the spline surface and the non-parametric distortion model, and a complete micro-vision imaging model was established based on a perspective projective model. Finally, the calibration calculation and optimization for the imaging model were performed by using the bundle adjustment algorithm. To validate the performance of the proposed method, a stereo microscope setup was developed for small-scale deformation measurement. By using an electronically controlled displacement setup, experiments for microscopic calibration and displacement measurement were conducted. Experimental results demonstrate that the focal length and the relative orientation parameters of the internal optics of the setup is obtained, and the displacement measurement accuracy is better than 1%. It concludes that the proposed method satisfies the accuracy requirement of the 3D deformation measurement during bulging processes and is also suitable for other microscopic vision measuring fields.