Aiming at the working characteristics of the end-effector of the gantry robot with only three translational degrees of freedom, a method for calibrating the point cloud coordinate system of the 3D vision sensor and the tool coordinate system of the robot actuator is designed, on the basis of the traditional two-step method of hand-eye calibration. In this method, only three calibration target pictures and three sets of point clouds are collected by two orthogonal translation movements of the robot, the rotation matrix and translation vector of the hand-eye relationship can be calibrated by measuring the base coordinates of mark points on the target through the TCP contact of the actuator. The method is simple to operate, and the calibration target is easy to make with low cost. The XINJE gantry robot and 3D vision sensor of structured light was used to build an experimental platform for experiments. The results show that the method has good stability and is suitable for field calibration, with calibration accuracy within ?0.2 mm.