In distributed optical fiber acoustic sensing(DAS) system, the birefringence of single-mode optical fibers results in the polarization state change of the output light in the optical fibers, which is not matched with the polarization state of the reference light, resulting in the polarization fading phenomenon. This paper designs a DAS system based on polarization diversity technology. The Rayleigh scattering light interferes with the reference light in two polarization states, and then photo-electric conversion and data acquisition are carried out. The polarization fading phenomenon is removed, and therefore the false alarm rate of DAS system is reduced, which makes the system much practicable for real applications. Experiment results show that in DAS system, the polarization diversity technology can eliminate the system misjudgment caused by polarization fading phenomena along the fiber, and can improve the alarm accuracy of the DAS system.