Aiming at the technical shortcomings of the current handheld Raman micro spectrometer, such as narrow spectral range, poor spectral resolution and large volume, a design method of the spectroscopic system for the handheld Raman spectrometer with high-resolution and wide-spectrum was proposed. On the basis of the comprehensive consideration of installation, debugging and performance, the wedge-shape cylindrical mirror was innovatively used to eliminate the astigmatism, which not only ensured the signal strength, but also facilitated the mechanical installation. Under the premise of restricting the change of the outgoing beam aperture after grating diffraction at different wavelengths, the aberration characteristics of the crossed asymmetric Czerny Turner optical path structure was used to achieve the coma correction in the full spectrum range. Based on the ZEMAX software, the spectrometer structure was simulated and optimized, and the prototype was developed. The size of the prototype was 72mm×43mm×62mm. The performance was tested by the helium-neon lamp. The results showed that the resolution was better than 6cm-1 and the Raman spectral range was 150~4000cm-1, which verified the feasibility and rationality of the optical design of the micro-spectrometer system.