Real-time target detection is one of the important research directions of infrared imaging spectral systems. In order to guarantee the data processing speed and spectral reproducing accuracy of the system at the same time, a high-speed spectral inversion system was studied. The system was implemented by a field-programmable gate array (FPGA) chip. Non-uniformity correction and windowed toe-cutting were applied to the interference fringe image to suppress the direct current noise and spurious noise in interference fringe data. Then spectral distribution was obtained after fast Fourier transform, phase correction and spectrum calibration. The results show that the algorithm has a good suppression effect on spurious noise. Coefficient of inhomogeneity decreases from 11.23% to 1.05%. In the experiment of spectral inversion, the spectral distribution obtained by this system is basically consistent with that obtained by MATLAB. The accuracy of spectral details is better. The system uses pipeline mode to shorten the data processing cycle. And the development module based on FPGA chip has better compatibility. The system has the advantages of fast processing speed, small volume, high stability and good compatibility. It has good application prospect in the field of infrared target real-time detection.