As the core element of various surface acoustic wave (SAW) devices, the performance of SAW resonator determines various specifications of SAW devices, among which the quality factor (Q) of resonator is the most important. The factors affecting the Q value of SAW resonators are introduced in this paper. It is pointed out that most of the substrate structure of SAW resonator with high Q is heterogeneous acoustic layer structure, and the heterogeneous acoustic layer structure is essentially one-dimensional phononic crystal. The results show that the Q value of the resonator can be improved by constructing phononic crystals at multiple positions to fully 3-D constrain the acoustic field energy of SAW resonator. Finally, the possible ideal structure of SAW resonator with high Q in the future is prospected.