In the photoelastic modulator-Fourier transform spectrometer (PEM-FTS), because the modulation optical path difference is nonlinear, the fast Fourier transform (FFT) cannot be directly used for spectrum recovery, and direct computing cost is too much. The phase of PEM interference signal is simulated by Matlab with the non-uniform fast Fourier transform algorithm (NUFFT), and then the core of the spectrum information processing system is designed with TMS320C6713 floating-point digital signal processer (DSP), finally, it realized the real-time processing of the spectrum on hardware. The result shows that the algorithm for spectrum recovery has some advantages, such as fast running speed, high accuracy and so on. 1024 point spectrum recovery speed is 20 times more than direct operation speed, recovery precision can reach 0.78%.