A optical time domain reflectometry (OTDR) system based on LabVIEW is designed. A 1625 nm wavelength pulsed laser module is used as the light source. The high-sensitivity avalanche diode photodetector and high-speed data acquisition card are used to collect, process and storage the optical signal under the control of the LabVIEW integrated virtual instrument development environment, in order to realize on-line real-time monitoring of fiber. An algorithm combining digital average method and weighted sliding-average is adopted to reduce the noise of the returned backscatter signal, and the effect of the digital average time, smooth width and smooth type on the result of signal processing of the OTDR system is quantitively analyzed. Then the best smooth model is given. The first derivative method is used and the double thresholds of derivative and amplitude are set to locate the point of the reflection event. The experimental results of practical optical cable line with the proposed system show that the optimal smoothing filter proposed in this paper improves the dynamic range by 1.1 dB and the signal-to-noise ratio by 1.25 times compared with the traditional smoothing filter at the same smooth width. The proposed positioning algorithm can be used to locate the point of the reflection event accurately, which indicates that the proposed algorithm has certain practical value.