A proper optical system is selected for a compact, wide waveband spaceborne hyperspectral imager by comparing with existing optical systems applied to spaceborne hyperspectral imagers. Firstly, the advantages and disadvantages of optical systems for the hyperspectral imagers are discussed. Then, the principle and characteristics of three spectral-splittering methods with a prism, a grating or FT(Fourier Transform) as dispersion elements are compared and analyzed. Finally, based on the research objective of a hyperspectral imager, a reflective optical system is chosen. The system is composed of an off-axis Three-Mirror Anastigmatic(TMA) telescope and two Offner convex grating spectrometers. By changing the magnification of two spectral imaging systems, two array detectors with different pixel sizes can be matched. The detailed design results are presented,which indicate that the Modulation Transfer Function (MTF) of 128 wavebands from 400 nm to 2 500 nm is all over 0.6, distortion is less than 0.44% and the spectral bend(smile) is less than 0.03%. The chosen reflective optical system can satisfy the technical requirements of the compact spaceborne hyperspectral imager and can efficiently decrease the volume and weight. By enlarging the relative aperture of the optical system, the low diffraction efficiency of gratings can be compensated.