In contrast to common data modalities such as visible light and infrared, hyperspectral images inherently offer advantages and stronger characteristics in target tracking tasks, holding great potential for applications in complex environments and scenarios. However, on the one hand, most improved correlation filter (CF) methods extract target features solely from spectral or false-color images, thereby resulting in insufficient target feature description. On the other hand, during the training process of spatially/temporally regularized correlation filters, such as kernelized correlation filter (KCF) and background-aware correlation filter (BACF), the varying sensitivity of different channels to background noise changes and the similarity of background features between adjacent frames are often overlooked. This leads to inadequate utilization of channel information and background environmental changes. These factors contribute to the decreased performance of CF algorithms when tracking targets in scenes with rapidly changing backgrounds, and tracking drift may even occur. To address the poor tracking performance of existing hyperspectral video tracking algorithms in scenarios with rapidly changing backgrounds, we propose an efficient correlation filter-based tracking algorithm to achieve robust tracking of moving targets in such environments.

An environmental residual-aware (ERA), multi-regularized correlation filter (MRCF) tracking algorithm is proposed in this study. First, to reduce computational complexity, a background-aware band selection method is employed to select three bands from multi-band hyperspectral images. The selected three-band images, characterized by the top three highest dissimilarity scores between the target and its local neighborhood, are formed into a three-channel spectral image for target tracking. Secondly, three typical features of the target in the false-color image and three-channel spectral image are extracted: histogram of oriented gradients (HOG), intensity, and 3D HOG features. These target features from the false-color image and the three-channel spectral image are fused by simply adding them in a naive manner to obtain the fused feature, thereby enhancing the ability to represent the target. Finally, the fused target feature is used as input for the improved MRCF to predict the target position. In the training stage of the CF, an ERA regularization term is introduced into the ridge regression optimization function of MRCF to suppress interference caused by rapid background changes.

To verify the effectiveness of the proposed algorithm, we select four hyperspectral target tracking algorithms and compare them in the experiment. Meanwhile, we select three specific sequences from the test set to visualize the performance of the proposed algorithm in comparison with the other four algorithms. We evaluate the algorithms from two aspects: tracking accuracy and success rate. Figure 3 shows the precision and success rate curves of each algorithm on the test sequence. Figures 4 and 5 demonstrate the precision and success rate in the presence of fast target motion and scale variation challenges. As shown in Fig. 3, the proposed algorithm ranks first in terms of precision and success rate on the total test sequence. Specifically, the precision increases by 2.51%, and the success rate grows by 1.67% compared to MHT. Due to the joint utilization of feature fusion and ERA regularization modules, the algorithm exhibits strong robustness. As shown in Fig. 4, in the case of tracking a target with fast motion, the precision and success rate of the proposed algorithm are much higher than those of the second-place algorithm. This occurs as the two modules—false-color/spectral feature fusion and ERA regularization—collaboratively reduce the effect of background changes between adjacent frames on the tracker, in terms of feature representation and filter training. Additionally, as shown in Fig. 5, under the challenge of scale variation, the precision and success rate of the proposed algorithm are 6.24% and 2.52% higher than those of the second-place algorithm, respectively, demonstrating excellent adaptability. Fig. 9 shows the qualitative analysis results of various algorithms in the selected sequences. In the Droneshow2 sequence, a small-sized UAV with low contrast against the surrounding background is flying from right to left. Since the proposed algorithm enhances its discriminative capability for small targets by using fused features from false-color/three-band spectral images, it can successfully discern the location of the UAV. In the L_car2 sequence, a person is walking from near to far, with variations in both target scale and background environmental information occurring during the process. The proposed algorithm incorporates a background ERA regularization term to effectively adapt to variations in the background environment and achieve tracking robustness. Table 2 presents the precision and success rate of the ablation experiment and reveals that the proposed method improves the algorithm’s robustness.

1) To address the issue of insufficient target feature description, which leads to inadequate target discrimination ability in hyperspectral object tracking tasks, we employ a fusion method based on the simple addition of features from false-color images and three-band spectral images. This method effectively balances the retention of details from each feature while reducing the effect of background or interfering features, thereby enhancing the representation capability of hyperspectral target features. 2) Furthermore, to tackle the problem of insufficient utilization of channel and background environmental change information, which leads to decreased tracking performance in rapidly changing background scenarios, we propose an ERA-MRCF. The proposed algorithm incorporates an ERA module within the MRCF framework, which, while preserving MRCF’s robust perception of target appearance changes, suppresses the interference caused by rapid background changes. This enhancement improves the tracker’s robustness in challenging scenarios such as fast target motion. Experimental results on the public hyperspectral datasets HOTC2024 and IMEC25 validate the algorithm’s excellent tracking performance in terms of fast motion and scale variation. 3) Future work will focus on improving band selection methods and feature representation for small hyperspectral targets. This includes not only fully exploring the fused feature representation for small targets, such as employing weighted fusion based on both overlap ratio and distance reliabilities, as well as deep learning-based multi-modality feature fusion to enhance the tracker’s ability to discriminate small targets, but also refining band selection methods to go beyond merely utilizing the dissimilarity information between the target and its surrounding background in the current frame. Additionally, efforts will be made to design intelligent fusion strategies for detection and tracking results to improve the algorithm’s robustness against occlusion.