To reduce the impact of random errors on hand-eye calibration in the visual system of a blade repair robot, an optimization method based on outlier detection is proposed. Firstly, a linear equation for the hand-eye matrix is established. The initial hand-eye matrix is solved using singular value decomposition (SVD). Secondly, the initial value is used to perform an inversion operation on the samples. Outlier samples are detected and removed based on Z-scores, leading to a more accurate hand-eye matrix. Finally, the obtained hand-eye matrix is used as the initial value for optimization. The rotation is represented by unit quaternions, and the Levenberg-Marquardt algorithm is applied to further optimize the initial value, yielding the final hand-eye matrix. Hand-eye calibration experiments were conducted on the blade repair robot equipped with a stereo depth camera. The real coordinates of the target points were obtained using a TCP calibration tool. The predicted coordinates from the hand-eye matrix, obtained by the proposed method, have an average Euclidean distance of 0.858 mm from the true coordinates, with a variance stabilizing below 0.1. Compared to other methods, the proposed approach effectively reduces the impact of random errors and demonstrates good stability and accuracy.