In the spectral domain optical coherence tomography system, the linear-wavenumber spectrometer can not only reduce the interpolation error of the image and improve the image quality, but also improve the imaging speed and sensitivity of the system. Therefore, in view of the shortcomings of the existing grism spectroscopic optical system, a method combining grism structure spectroscopy and object image aberration compensation is proposed in this paper. By introducing distortion and vertical axis chromatic aberration, extremely high wavenumber linearity is achieved. A linear-wavenumber spectrometer with a working band of 750--950 nm, a spectral resolution of 0.1 nm, and a length of the system is about 148 mm is designed. The design results show that the wavenumber linearity of the spectrometer reaches 0.0056, which is about 11 times that before aberration compensation. Simulation analysis results show that the spectroscopic system can significantly improve the sensitivity of the optical coherence tomography system. Compared with the uncompensated system, the sensitivity of the compensated system at an optical path difference of 3.5 mm is increased by 7.8 dB.