Based on the geometrical property of orthogonal vanishing point and the infinite homograph relationship between corresponding vanishing points in different camera positions of the camera image plane, a camera intrinsic parameter self-calibration method is proposed. The proposed method takes in two sets of orthogonal parallel lines under any two positions, and four vanishing points in two images are achieved and decision algorithm is designed to determine their corresponding situations. Using the infinite homograph relationship between corresponding vanishing points, the constraint equation of orthogonal vanishing points and camera optical center line are established to realize the solution of the camera intrinsic parameters linearly. Aiming at restrain the influence of noise to the calibration result, the indicator function based on the inverse points of rectangle imaging is proposed, using LM optimization algorithm to achieve the intrinsic parameters of the nonlinear optimization. Experimental results show that the proposed method can judge corresponding vanishing points simply and effectively, the calibration results has high precision, strong real-time performance and wide application scope. The robustness of calibration algorithm is improved through parameter optimization to reduce the image noise.