In the scenario of multi-target tracking of infrared UAV swarm，there are challenges such as limited appearance features and severe target homogeneity，mutual occlusion of individuals in the swarm and platform jittering. To address the problems，this paper proposes a fusion tracking algorithm based on Graph Convolutional neural Network (GCN) and clustering algorithms. Firstly，a self-attention feature mask is introduced to enhance GCN’s trajectory aggregation. Secondly，IoU and likelihood-based C-means clustering are adopted to improve motion feature extraction and adjacent target differentiation in the swarm. Finally，optimization of the tracking results is further achieved through a trajectory connection model and a Gaussian smoothing interpolation algorithm. The proposed algorithm integrates short-time trajectory aggregation and long-time trajectory matching，and achieves infrared UAV swarm multi-target tracking by only using motion information and interaction information. The experiments are conducted on infrared UAV swarm multi-target tracking dataset. The experimental results demonstrate its superior performance compared with that of other advanced tracking algorithms. MOTA and IDF1 of the proposed algorithm reach 84.9% and 80.2% respectively，and it has excellent tracking effects even in complex scenarios such as mutual occlusion of targets and platform jittering.