In order to reduce the volume and weight of the underwater Raman system, and to improve its portability and detection sensitivity, a compact underwater Raman spectroscopy system with high sensitivity was developed and assessed. Through elaborate selection of components, a compact structural design was realized with both the weight and the volume well controlled. The size of the main body was kept at 795 mm in length and 260 mm in diameter, with a weight of 548 N, one third of the weight of reported DORISS (the first deep ocean Raman in-situ spectroscopy system). The laser was housed in the optical probe rather than in the main body, hence higher excitation efficiency was achieved with high power density. There are two advantages to put the laser head in the probe. A desirable excitation power density could be obtained without the consumption of laser beam during transmission in fiber, and better signal to noise ratio could be achieved without the stray light raised by the interaction of laser and optical fiber. In addition, 300 mW powered laser, efficient volume phase holographic grating and TEC cooled CCD detector were used to improve the system performance. The experimental results show that the LOD (limit of detection) of SO2-4 was less than 0.4 mmol·L-1. Its about four times than the value achieved by DORISS. Meanwhile the system can be used to identify minerals. All above prove the system to be highly potential in ocean exploration.