OPTICS & OPTOELECTRONIC TECHNOLOGY, Volume. 20, Issue 2, 26(2022)

Key Point Guided Phase Measurement for 3D Facial Expression Acquisition

SONG Ren-jie1,2, ZHANG Guang-hui1,2, YING Xiao-lin1,2, and LI Yong1
Author Affiliations
  • 1[in Chinese]
  • 2[in Chinese]
  • show less

    3D facial expression capture is widely used in film and television, game, medicine, social intercourse and other fields. In order to realize high-precision acquisition of 3D facial expression information in unmarked free movement, a fringe projection 3D face measurement method based on key point detection and matching is proposed. Firstly, the quasi-static 3D information of the object to be measured is obtained as a reference model by using three-frequency heterodyne method, and the initial key points is obtained from the modulation image by using the face key point detection algorithm based on deep learning.The 3D coordinates corresponding to these key points are then obtained. During the dynamic measurement, only two or three fringes with phase shifts are projected. Modified Fourier transform profilometry or phase measurement profilometry is used to obtain the wrapped phase and modulation of deformed fringe. Face key points are detected in the modulation image and matched with the initial key points to obtain the coordinate relationship between the key points and the reference model. The motion parameters of the face relative to the reference model are calculated and the approximate phase is further calculated using the reference model. Finally, the approximate phase is used to unwrapthe phase and reconstruct the 3D shape to obtain the 3D facial expression at that time. A 3D facial expression capture system is built by using DLP projector and high-speed camera. 3D data acquisition with 1.3 million pixels at 100 fps is realized. The experimental results show that the system can accurately capture the unmarked 3D expressions with significant changes within ±45 degrees of deflection, ±30 degrees of pitch and ±45 degrees of roll.

    Tools

    Get Citation

    Copy Citation Text

    SONG Ren-jie, ZHANG Guang-hui, YING Xiao-lin, LI Yong. Key Point Guided Phase Measurement for 3D Facial Expression Acquisition[J]. OPTICS & OPTOELECTRONIC TECHNOLOGY, 2022, 20(2): 26

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category:

    Received: Sep. 6, 2021

    Accepted: --

    Published Online: Aug. 2, 2022

    The Author Email:

    DOI:

    CSTR:32186.14.

    Topics