The spectrum of amplifier spontaneous emission (ASE) is non-Gaussian, sidelobe appears inevitably and it will infect the measurement of molecular which is more or less the same as the coherence length seriously. In this paper, the denoising function of wavelet transform is used, and the coherence signal is processed with Daubechies 10 making three layer wavelet decomposition, different frequency signals are denoising by different threshold values. It can not reduce the influence of sidelobe greatly, but remain most of the mainlobe signal. The result shows that wavelet transform is good for extracting the useful information. A particle specimen is used to simulate the measurement of molecular and the diameter is about 30 μm. The results of the experiments demonstrate this system has the advantage of high resolution.