Large-scale hyperspectral remote sensing monitoring is an important means of environmental supervision in rare earth mining areas, and the analysis of characteristic variation of reclaimed vegetation under environmental stress in mining areas provides a necessary basis for accurate dynamic monitoring of ecological restoration in mining areas. The original spectra of six typical reclaimed vegetation and their corresponding normal environment vegetation leaves in rare earth mining areas were collected on the spot, and their spectral variations were compared and analyzed. In addition to subjecting the original spectra to the usual derivative transform (DT), fractal dimension (FD) calculations in signal processing, discrete wavelet transform (DWT) analysis techniques, and short-time Fourier transform (STFT) processing are applied to amplify the detailed information of vegetation leaf spectra to investigate the spectral characteristics of reclaimed vegetation under environmental stress in the rare earth mining area. The results show that: (1) In the first-order derivative spectra, all vegetation except wetland pine show a blue shift in the “red edge position”, indicating that the reclaimed vegetation is affected by external factors such as environmental stress to varying degrees in the mining area. (2) By calculating the FD of vegetation spectral curves in mining areas, the FD of the same species of reclaimed vegetation is higher than that of normal vegetation, indicating that the influence of multiple conditions of environmental stress in mining areas will cause the waveforms of reclaimed vegetation spectral curves to become complex. (3) The vegetation leaf spectra are discrete wavelet transformed, where the best detail coefficient of the original spectral DWT is d5, the best detail coefficient of the first-order derivative spectral DWT is d6, and the first-order derivative spectral DWT amplifies the difference in spectral feature details at a smaller scale, achieving better results. (4) The spectra are localized in the null-frequency diagram by the STFT, with the original spectral null-frequency features appearing at the “red edge” and the first “trough” in the mid-infrared, while the first-order derivatives amplify and increase the spectral curve null-frequency features at smaller scales and in more bands. In general, applying signal processing methods to spectral processing can obtain more spectral features than the DT, where the STFT is superior to FD calculation and DWT analysis techniques in terms of spectral mapping into null-frequency features. The study results provide technical support for the inversion of physiological parameters of the reclaimed vegetation in rare earth mining areas and the monitoring of reclamation effects, which will help the ecological reconstruction of reclaimed mining areas.