A novel camera calibrating method based on orthogonal vanishing points is proposed after analyzing camera model, calibration of single and stereo vision systems and geometrical characteristics of the vanishing point. Only one camera photographing at least three checkerboard patterns in different orientations is required. The pinhole and the first two terms of radial distortion model is introduced. The presented method mainly contains the computation and optimization of vanishing points, a linear calibration of intrinsic parameters, the improved Tsai′s two-steps for extrinsic parameters and a nonlinear global refinement. Experimental result demonstrates that the proposed algorithm is fast and accurate. After global optimization, the re-projective mean pixel error is 0.0043 pixel and the time consumption is 1.7911260 s. This novel method can be widely applied in fields including the computer vision research, the industry three-dimensional measurement and reconstruction and road mapping.