The micro-vibration sensor based on cascaded grating is a typical micro-vibration signal measurement scheme. However, due to the multiple transmission and reflection of optical signal in the cascaded grating, the sensor is subject to the poor spectral signal-to-noise ratio and complex components. Based on this, a spectrum signal optimization algorithm combined with empirical mode decomposition and chebyshev filter was proposed in this paper. Firstly, the original spectrum of the sensor was decomposed into a series of intrinsic eigenmode functions by empirical mode decomposition; Secondly, the order of the intrinsic mode functions including the reflection peak component was determined by using the proposed adaptive filtering method, and the chebyshev low-pass filtering was performed on these orders; Finally, the optimized sensor spectrum was obtained by reconstructing the output of the filter. A micro-vibration excitation signal with an amplitude of ±8 mV and frequency of 500 Hz was used for experimental verification. The results show that the proposed algorithm can effectively restore the micro-vibration signal from the excitation source, and the accuracy is improved by more than 87.5% compared with the traditional methods.