Realizing real-time and highly accurate three-dimensional (3D) imaging of dynamic scenes presents a fundamental challenge across various fields, including online monitoring and augmented reality. Currently, traditional phase-shifting profilometry (PSP) and Fourier transform profilometry (FTP) methods struggle to balance accuracy and measurement efficiency simultaneously, while deep-learning-based 3D imaging approaches lack in terms of speed and flexibility. To address these challenges, a deep-learning-assisted real-time method has been proposed for real-time 3D imaging in dynamic scenes.

The several stages of the 3D imaging pipeline have been optimized for high accuracy, efficiency, and flexibility. For fringe projection, by determining an appropriate projection resolution for fringes, we effectively increased the number of fringe periods within the tested target (i.e., ROI) and thereby improved phase accuracy. For phase demodulation, a phase estimation module was designed to provide reliable initial phases for the subsequent lightweight neural network, resulting in the phase-estimation network (PE-Net). In formulating the loss function, the similarity of information in both spatial and frequency domains was considered for effectively improving the accuracy and speed of the phase prediction. For phase unwrapping, theoretical analysis was employed for the MHPU method from the perspective of depth constraint, resulting in a more rational selection of frequencies.

The optimization is conducted at different stages of the 3D imaging process, which allows for one 3D reconstruction result for each newly captured image. The entire pipeline achieves real-time 3D imaging with an RMS error of less than 0.031 mm, a resolution of 1280ⅹ800, and a speed over 100 frame/s, providing a more efficient lightweight solution for dynamic scene measurement. The detailed principle analysis and result presentation is published entitled "Real-time 3D imaging based on ROI fringe projection and a lightweight phase-estimation network" in Advanced Imaging.

Flowchart of real-time 3D imaging based on ROI projection and phase estimation.