In order to broaden the scope of application and ensure the calibration precision, a new method of linear calibration by making fully use of vanishing point attributes is proposed. The method dose not need any rigorous restrictions, and solves the self-calibration problem with only five vanishing points in two digital images, which are arbitrarily taken by a handheld digital camera. Furthermore, another approach for camera's pose estimation is also put forward without any strict controlled motions. The experimental results of both computer simulation and real images show that the calibration algorithm is effective, feasible, and robust.