The sensitivity of the phase-sensitive optical time-domain reflecting (Φ-OTDR) system is limited by the system’s intrinsic noise, such as the laser’s phase noise, the erbium-doped fiber amplifier’s spontaneous emission noise, and the photodetector’s shot and thermal noise, as well as random environment of noise. Therefore, we investigate the noise reduction algorithm based on the optical time-domain reflecting data to improve the system’s signal-to-noise ratio (SNR) without degrading its frequency response range. Furthermore, we propose a Savitzky-Golay smoothing algorithm by selecting a slidable fixed-length window to process OTDR data for the SNR improvement while maintaining the system sampling frequency. Then, we built the experimental system to demonstrate the results. The experimental results show that by using the Savitzky-Golay smoothing algorithm, the SNR of the system is improved by 5.41 dB relative to the difference method with the original data, and the SNR is improved by 3.39 dB and 5.05 dB, compared to the commonly used cumulative averaging method and sliding averaging method, respectively. It is demonstrated that the Savitzky-Golay smoothing algorithm can improve the sensitivity and accuracy of the Φ-OTDR system, which helps to sense weak vibration events accurately and reduce false alarm rates.