In coherent optical communication system, the phase noise of laser sources has been identified as a crucial characteristic that affects the receiver sensitivity of optical communication terminal. To characterize the laser sources phase noise of coherent optical communication system, three critical parameters which contained the field spectrum, phase-error variance and FM-noise spectrum were proposed for noise test in this paper. A delayed self-homodyne phase noise measurement system with optical coherent receiver was established and the optical communication system simulation had been achieved at the same time. The phase noise tests based on delayed self-homodyne phase noise measurement system which were used to evaluate the quality of coherent optical communication system had been performed on a narrow line-width semiconductor laser. Compared with traditional self-heterodyne narrow line-width measurement, this method can fully characterize the laser phase noise and achieve the requirement of resolution.