To solve the problem that UAV target tracking is difficult to achieve robust tracking due to its vulnerability to perspective changes, partial occlusion, background clutter and other factors, a UAV target tracking algorithm with double response anomaly constraints is proposed. Firstly, by calculating the local response change vector and global response peak side lobe ratio, the algorithm dynamically updates the spatial regularization parameters online to reduce the influence of boundary effect. Secondly, aiming at the problem that the parameters of the time regularization term in the STRCF algorithm are fixed, the judgment mechanism of abnormal response peak is proposed, the sub-peak response coefficient is calculated, and it is fused into the time regularization parameter for real-time update, so as to avoid the tracking drift problem caused by the target template variation. The objective function is iteratively solved by alternate direction multiplier method. A large number of experiments are conducted on UAV123, UAV20L and DTB70 UAV data sets and with representative target tracking algorithms. The results show that the accuracy of UAV123 data set reaches 71.7%, and the average tracking speed of DTB70 data set reaches 48.3 frame/s. Compared with the existing UAV target tracking algorithms based on correlation filtering, the algorithm proposed in this paper has better performance, high real-time performance and robustness, and can better complete UAV target tracking.