Visual object tracking based on deep learning has become one of the mainstream algorithms in the target tracking field due to its performance. The main idea of algorithms is to learn the similarity of the previous and next frames to complete the matching of the template frame and the search frame. Among them, similarity learning is a key step that affects the performance of tracking algorithms. This paper takes the similarity learning of siamese network as an entry point, improves the similarity learning method of deep-wise cross correlation (DW-XCorr), and proposes a target tracking algorithm for multi-scale similarity learning.Under the basic network framework of SiamRPN, this algorithm applies a multi-scale cross correlation (MS-XCorr) module, which improves the crosscorrelation operation in multi-scale, thereby increasing the diversity of learning feature scales and improving the similarity of the tracking network The efficiency of similarity learning ultimately further improves the tracking performance of the algorithm.In the experimental part, the improved algorithm is compared with its baseline: the algorithm has improved success rate, precision and norm precision, and the success rate has increased. 4.3%, the accuracy increased by 4.4%, and the average accuracy increased by 4.0%. Experiments show that the multi-scale cross correlation module has stronger similarity learning ability than the deep-wise cross correlation module.The proposed multi-scale similarity learning target tracking algorithm has better tracking performance in complex situations such as target illumination, morphological changes, occlusion and interference.