In the practical application of camera pose estimation algorithm, affected by the matching error, the reference points often contain outliers. In view of the weak ability of existing algorithms to resist the outliers, a binary weighted algorithm based on orthogonal iteration is proposed. Based on the orthogonal iterative algorithm, a weighted system with two thresholds is introduced to measure the reliability of the reference point. The maximum value of the quartile of the reprojection error is selected as one of the thresholds. In addition, a threshold related to the reprojection error and focal length is introduced to accelerate the convergence speed when the proportion of outliers is low. Finally, the maximum value of two thresholds is selected to weight the objective function, which reduces the influence of outliers on the estimation. The experimental results show that the method improves the anti-outlier ability of the orthogonal iterative algorithm and performs better in robustness.