In view of the problem that commercial spectrometers can not realize spectral detection in the extremely weak light measurement environment, this paper puts forward a kind of spectral measurement system based on single photon detector array, using the single photon detector in a very weak light environment detection ability, the virtual image phase array(VIPA) and reflective diffraction grating of two dimensional dispersion structure are combined with single photon detector array. The two dimensional effect, relative spectral transmittance curve and single photon detection performance of the system are obtained by experimental test. The experimental results show that the system not only has high sensitivity, but also achieves wavelength resolution of 0.006 nm. Compared with the widely used spectral measurement system based on time-dependent single photon counting, the proposed system has better wavelength resolution and larger measurement bandwidth.