In this paper, to address the problem that the single-object tracking algorithm of Siamese fully convolutional networks cannot extract the high-level semantic features of the object and cannot focus on and learn the channel, spatial and coordinate features of the object at one time, which leads to degradation of the tracking performance and tracking failures when faced with the challenges of the object's deformation, attitude changes, and background interference in a complex scenario, we propose a single-object tracking algorithm for Siamese networks based on the multiple-attention mechanism and response fusion. In this algorithm, three modules, namely, the backbone feature extraction network with small convolutional kernel fused with jump-layer connected features, the improved attention mechanism, and the response fusion operation after convolutional inter-correlation are designed to enhance the tracking performance of this algorithm, and the effectiveness of these three modules is verified by ablation experiments. Finally, after testing on the OTB100 benchmark dataset, the tracking accuracy reaches 0.825, and the tracking success rate reaches 0.618. Meanwhile, compared with other advanced algorithms, it shows that the algorithm not only can effectively cope with the problem of decreasing performance of object tracking algorithms in complex scenarios, but also can further improve the tracking accuracy under the premise of guaranteeing the tracking speed.