Hyperspectral images capture spectral information across dozens or even hundreds of bands at each pixel location, far exceeding the three bands of RGB images, thereby providing richer and more detailed spectral data. By leveraging the abundant spectral information contained in hyperspectral images, limitations of current visible light-based target tracking methods under complex scenarios, such as background clutter, illumination variations, and motion blur, can be overcome. To address the challenge of background clutter in hyperspectral target tracking, a siamese fusion hyperspectral target tracking method based on spectral segmentation was proposed.Due to the high dimensionality and large volume of hyperspectral images, direct processing would consume significant computational resources, necessitating appropriate dimensionality reduction to mitigate data redundancy while preserving crucial spectral information. In the first frame of the video, the target and search regions are manually defined. Initially, spectral deviation segmentation is employed to obtain the target spectral curve, which is then utilized in spectral angle distance computation for dimensionality reduction. Following this, the reduced-dimensionality images are individually fed into a Semantic Parsing Network (SP-Net) and an Appearance Descriptor Network (AD-Net) for the extraction of deep and shallow features, respectively. Shallow features, often sensitive to subtle changes, capture detailed information such as texture and color, which are vital for recognizing the target's appearance. Conversely, deep features focus more on high-level semantic information like shape and category, providing more stable and discriminative features that aid the model in distinguishing targets from backgrounds in complex scenes. By combining AD-Net and SP-Net, the advantages of both are fully leveraged, enhancing tracking robustness and accuracy. The extracted target region features and search region features undergo cross-correlation to generate two initial response maps, which are then fused using a peak-to-correlation energy-based response map fusion approach. The final response map determines the target's location.To validate the effectiveness of the proposed algorithm in this paper, six trackers were selected as comparative algorithms, including four hyperspectral target tracking algorithms and two RGB target tracking algorithms. This paper presents the experimental results in image form. From the qualitative analysis diagram in Fig.5, it can be observed that the proposed algorithm possesses strong anti-interference capabilities, effectively dealing with noise and distracting information in complex backgrounds, thereby ensuring stability and reliability of target tracking in dynamic scenes. Figures 6-8 respectively demonstrate the success rate and precision of each algorithm on the test sequences, as well as their performance under background clutter and illumination variation challenges. Tables 1 and 2 correspond to the accuracy and success rates of the proposed algorithm and the comparative algorithms. As seen from Tables 1 and 2, the proposed algorithm exhibits excellent tracking performance. It achieves a success rate of 63.8% on all test sequences, ranking first and surpassing the second-ranked algorithm by 0.23%. In terms of precision, it scores 84.78%, ranking second and only 0.04% lower than the top algorithm. Overall, the proposed algorithm remains competitive. Under the BC challenge, by combining shallow appearance information with deep semantic features, the algorithm achieves a success rate of 69.98%, surpassing the second-best by 4.48%, and a precision of 89.66%, outperforming the second-ranked algorithm by 0.59%. For the IV challenge, the algorithm demonstrates robust generalization capabilities, with a success rate of 60.53%, 7.16% higher than the second-best, and a precision of 83.52%, 0.48% higher than the second-ranked algorithm. Table 3 presents the results of ablation experiments, indicating that the proposed algorithm with all modules preserved demonstrates superior performance.Addressing the challenge of background clutter in hyperspectral video target tracking algorithms, this paper proposes a hyperspectral target tracking algorithm based on spectral deviation segmentation and dual-siamese fusion. This algorithm utilizes spectral deviation segmentation to reduce data redundancy, separate the target from the background, and obtain the target spectral curve for dimensionality reduction. AD-Net and SP-Net are employed to extract appearance detail information and deep semantic information, respectively. A dual-siamese network is then adopted to generate two target response maps, which are fused using average peak-to-correlation energy, enabling target tracking. Experimental results demonstrate that the proposed algorithm outperforms other algorithms in terms of target tracking stability under background clutter challenges, effectively addressing this challenge. Furthermore, future work will explore the use of the current advanced Transformer framework to tackle scale deformation challenges in hyperspectral target tracking, thereby enhancing the generality and adaptability of the tracking algorithm.