The uncooled thermal infrared imaging spectrometer with small size, low cost, and long life has great potential for detecting high-temperature targets, but its severe internal noise will reduce the detection sensitivity. The main noise of uncooled thermal infrared imaging spectrometer includes the detector noise, the noise caused by the stray light from the surface of the opto-mechanical component, stray light noise generated by the non-working order of the grating, and the background radiation noise of the opto-mechanical component. The noise equivalent temperature difference (NETD) formula containing the above noise is derived. Taking the Offner thermal infrared imaging spectrometer as an example, the relationship among NETD, the F number of the system, and the optical properties of the inner surface of mechanical components is analyzed. Then, the ability of opto-mechanical surface polishing to suppress the internal background radiation noise of the spectrometer is studied. Finally, the linear relationship between pixels is used to remove the remaining noise, so that the detection signal is basically consistent with the target signal.