Rapidly downsized feature size of lithography process in integrated circuit (IC) manufacturing brings critical demand on smaller overlay error. Correspondingly, overlay metrology and system with a sub-nanometer accuracy are becoming significant requirement. Thus, this study introduces two existing typical measurement techniques, including diffraction-based overlay (DBO) and image-based overlay (IBO), and measurement principles and characteristics of each technology are presented. More intensive review of DBO with higher precision measurement capability is taken, in this part recent progress, challenges, and future works are involved. Hopefully, this study can provide technical references for research and development of home-made advanced lithography machines.