To visually detect a target ball for a laser tracker in a complex scene, a method of target ball detection improved by deep learning was proposed. Firstly, the characteristics of the target ball, its application environment, and its effect in tracking recovery were analyzed. Subsequently, Hypernet and shallow high-resolution features were adopted, New feature maps and an optimized region proposal were added to the original network, improving the network sensitivity to enable the detection of multi-scale and small targets. Hard example mining with strong background interference was used to reduce the ratio of error recognition, which resulted from similar objects. Finally, the dataset was established and a comparative experiment was carried out. The experiment results show that the improved method proposed in this study and hard example mining with strong background interference can increase the correct recognition rate obtained by Faster R-CNN, yielding a value of 90.11% in the test and meeting the tracking recovery requirement.