Photonics Research, Volume. 9, Issue 6, 1084(2021)

Generalized framework for non-sinusoidal fringe analysis using deep learning

Shijie Feng1,2,3、*, Chao Zuo1,2,4、*, Liang Zhang1,2, Wei Yin1,2, and Qian Chen2,5、*
Author Affiliations
  • 1Smart Computational Imaging (SCI) Laboratory, Nanjing University of Science and Technology, Nanjing 210094, China
  • 2Jiangsu Key Laboratory of Spectral Imaging & Intelligent Sense, Nanjing University of Science and Technology, Nanjing 210094, China
  • 3e-mail: shijiefeng@njust.edu.cn
  • 4e-mail: zuochao@njust.edu.cn
  • 5e-mail: chenqian@njust.edu.cn
  • show less
    References(78)

    [1] K. Harding. Industrial metrology: engineering precision. Nat. Photonics, 2, 667-669(2008).

    [2] J. M. Schmitt. Optical coherence tomography (OCT): a review. IEEE J. Sel. Top. Quantum Electron., 5, 1205-1215(1999).

    [3] J. Han, L. Shao, D. Xu, J. Shotton. Enhanced computer vision with Microsoft Kinect sensor: a review. IEEE Trans. Cybern., 43, 1290-1334(2013).

    [4] P. Paysan, R. Knothe, B. Amberg, S. Romdhani, T. Vetter. A 3D face model for pose and illumination invariant face recognition. 6th IEEE International Conference on Advanced Video and Signal Based Surveillance, 296-301(2009).

    [5] M. M. P. A. Vermeulen, P. Rosielle, P. Schellekens. Design of a high-precision 3D-coordinate measuring machine. CIRP Ann., 47, 447-450(1998).

    [6] F. Chen, G. M. Brown, M. Song. Overview of 3-D shape measurement using optical methods. Opt. Eng., 39, 10-22(2000).

    [7] R. Leach. Optical Measurement of Surface Topography, 14(2011).

    [8] J. Geng. Structured-light 3D surface imaging: a tutorial. Adv. Opt. Photonics, 3, 128-160(2011).

    [9] J. Salvi, J. Pagès, J. Batlle. Pattern codification strategies in structured light systems. Pattern Recogn., 37, 827-849(2004).

    [10] C. Zuo, S. Feng, L. Huang, T. Tao, W. Yin, Q. Chen. Phase shifting algorithms for fringe projection profilometry: a review. Opt. Lasers Eng., 109, 23-59(2018).

    [11] L. Zhang, B. Curless, S. Seitz. Rapid shape acquisition using color structured light and multi-pass dynamic programming. 1st International Symposium on 3D Data Processing Visualization and Transmission, 24-36(2002).

    [12] M. Schaffer, M. Grosse, B. Harendt, R. Kowarschik. High-speed three-dimensional shape measurements of objects with laser speckles and acousto-optical deflection. Opt. Lett., 36, 3097-3099(2011).

    [13] S. Heist, P. Lutzke, I. Schmidt, P. Dietrich, P. Kühmstedt, A. Tünnermann, G. Notni. High-speed three-dimensional shape measurement using GOBO projection. Opt. Lasers Eng., 87, 90-96(2016).

    [14] M. Takeda, K. Mutoh. Fourier transform profilometry for the automatic measurement of 3-D object shapes. Appl. Opt., 22, 3977-3982(1983).

    [15] X. Su, Q. Zhang. Dynamic 3-D shape measurement method: a review. Opt. Lasers Eng., 48, 191-204(2010).

    [16] Q. Kemao. Two-dimensional windowed Fourier transform for fringe pattern analysis: principles, applications and implementations. Opt. Lasers Eng., 45, 304-317(2007).

    [17] J. Zhong, J. Weng. Spatial carrier-fringe pattern analysis by means of wavelet transform: wavelet transform profilometry. Appl. Opt., 43, 4993-4998(2004).

    [18] L. Huang, Q. Kemao, B. Pan, A. K. Asundi. Comparison of Fourier transform, windowed Fourier transform, and wavelet transform methods for phase extraction from a single fringe pattern in fringe projection profilometry. Opt. Lasers Eng., 48, 141-148(2010).

    [19] V. Srinivasan, H.-C. Liu, M. Halioua. Automated phase-measuring profilometry of 3-D diffuse objects. Appl. Opt., 23, 3105-3108(1984).

    [20] P. S. Huang, Q. J. Hu, F.-P. Chiang. Double three-step phase-shifting algorithm. Appl. Opt., 41, 4503-4509(2002).

    [21] P. Hariharan, B. Oreb, T. Eiju. Digital phase-shifting interferometry: a simple error-compensating phase calculation algorithm. Appl. Opt., 26, 2504-2506(1987).

    [22] S. Zhang, S.-T. Yau. High-speed three-dimensional shape measurement system using a modified two-plus-one phase-shifting algorithm. Opt. Eng., 46, 113603(2007).

    [23] P. Jia, J. Kofman, C. E. English. Two-step triangular-pattern phase-shifting method for three-dimensional object-shape measurement. Opt. Eng., 46, 083201(2007).

    [24] P. S. Huang, S. Zhang, F.-P. Chiang. Trapezoidal phase-shifting method for three-dimensional shape measurement. Opt. Eng., 44, 123601(2005).

    [25] T. Anna, S. K. Dubey, C. Shakher, A. Roy, D. S. Mehta. Sinusoidal fringe projection system based on compact and non-mechanical scanning low-coherence michelson interferometer for three-dimensional shape measurement. Opt. Commun., 282, 1237-1242(2009).

    [26] Y. Guan, Y. Yin, A. Li, X. Liu, X. Peng. Dynamic 3D imaging based on acousto-optic heterodyne fringe interferometry. Opt. Lett., 39, 3678-3681(2014).

    [27] S. Yoneyama, Y. Morimoto, M. Fujigaki, M. Yabe. Phase-measuring profilometry of moving object without phase-shifting device. Opt. Lasers Eng., 40, 153-161(2003).

    [28] C. Zuo, Q. Chen, G. Gu, S. Feng, F. Feng. High-speed three-dimensional profilometry for multiple objects with complex shapes. Opt. Express, 20, 19493-19510(2012).

    [29] S. Ma, C. Quan, R. Zhu, L. Chen, B. Li, C. Tay. A fast and accurate gamma correction based on Fourier spectrum analysis for digital fringe projection profilometry. Opt. Commun., 285, 533-538(2012).

    [30] K. Liu, Y. Wang, D. L. Lau, Q. Hao, L. G. Hassebrook. Gamma model and its analysis for phase measuring profilometry. J. Opt. Soc. Am. A, 27, 553-562(2010).

    [31] S. Zhang, S.-T. Yau. Generic nonsinusoidal phase error correction for three-dimensional shape measurement using a digital video projector. Appl. Opt., 46, 36-43(2007).

    [32] Z. Li, Y. Shi, C. Wang, Y. Wang. Accurate calibration method for a structured light system. Opt. Eng., 47, 053604(2008).

    [33] B. Pan, Q. Kemao, L. Huang, A. Asundi. Phase error analysis and compensation for nonsinusoidal waveforms in phase-shifting digital fringe projection profilometry. Opt. Lett., 34, 416-418(2009).

    [34] H. Guo, H. He, M. Chen. Gamma correction for digital fringe projection profilometry. Appl. Opt., 43, 2906-2914(2004).

    [35] T. Hoang, B. Pan, D. Nguyen, Z. Wang. Generic gamma correction for accuracy enhancement in fringe-projection profilometry. Opt. Lett., 35, 1992-1994(2010).

    [36] C. Jiang, S. Xing, H. Guo. Fringe harmonics elimination in multi-frequency phase-shifting fringe projection profilometry. Opt. Express, 28, 2838-2856(2020).

    [37] B. Li, Y. Wang, J. Dai, W. Lohry, S. Zhang. Some recent advances on superfast 3D shape measurement with digital binary defocusing techniques. Opt. Lasers Eng., 54, 236-246(2014).

    [38] H. Fujita, K. Yamatan, M. Yamamoto, Y. Otani, A. Suguro, S. Morokawa, T. Yoshizawa. Three-dimensional profilometry using liquid crystal grating. Proc. SPIE, 5058, 51-60(2003).

    [39] T. Yoshizawa, H. Fujita. Liquid crystal grating for profilmetry using structured light. Proc. SPIE, 6000, 60000H(2005).

    [40] G. A. Ayubi, J. A. Ayubi, J. M. Di Martino, J. A. Ferrari. Pulse-width modulation in defocused three-dimensional fringe projection. Opt. Lett., 35, 3682-3684(2010).

    [41] C. Zuo, Q. Chen, S. Feng, F. Feng, G. Gu, X. Sui. Optimized pulse width modulation pattern strategy for three-dimensional profilometry with projector defocusing. Appl. Opt., 51, 4477-4490(2012).

    [42] Y. Wang, S. Zhang. Optimal pulse width modulation for sinusoidal fringe generation with projector defocusing. Opt. Lett., 35, 4121-4123(2010).

    [43] J. Sun, C. Zuo, S. Feng, S. Yu, Y. Zhang, Q. Chen. Improved intensity-optimized dithering technique for 3D shape measurement. Opt. Lasers Eng., 66, 158-164(2015).

    [44] W. Lohry, S. Zhang. Genetic method to optimize binary dithering technique for high-quality fringe generation. Opt. Lett., 38, 540-542(2013).

    [45] S. Feng, L. Zhang, C. Zuo, T. Tao, Q. Chen, G. Gu. High dynamic range 3D measurements with fringe projection profilometry: a review. Meas. Sci. Technol., 29, 122001(2018).

    [46] S. Feng, Y. Zhang, Q. Chen, C. Zuo, R. Li, G. Shen. General solution for high dynamic range three-dimensional shape measurement using the fringe projection technique. Opt. Lasers Eng., 59, 56-71(2014).

    [47] S. Zhang, S.-T. Yau. High dynamic range scanning technique. Opt. Eng., 48, 033604(2009).

    [48] Z. Song, H. Jiang, H. Lin, S. Tang. A high dynamic range structured light means for the 3D measurement of specular surface. Opt. Lasers Eng., 95, 8-16(2017).

    [49] S. Feng, Q. Chen, C. Zuo, A. Asundi. Fast three-dimensional measurements for dynamic scenes with shiny surfaces. Opt. Commun., 382, 18-27(2017).

    [50] C. Waddington, J. Kofman. Saturation avoidance by adaptive fringe projection in phase-shifting 3D surface-shape measurement. International Symposium on Optomechatronic Technologies, 1-4(2010).

    [51] L. Zhang, Q. Chen, C. Zuo, S. Feng. High dynamic range 3D shape measurement based on the intensity response function of a camera. Appl. Opt., 57, 1378-1386(2018).

    [52] H. Lin, J. Gao, Q. Mei, Y. He, J. Liu, X. Wang. Adaptive digital fringe projection technique for high dynamic range three-dimensional shape measurement. Opt. Express, 24, 7703-7718(2016).

    [53] Z. Cai, X. Liu, X. Peng, Y. Yin, A. Li, J. Wu, B. Z. Gao. Structured light field 3D imaging. Opt. Express, 24, 20324-20334(2016).

    [54] V. Suresh, Y. Wang, B. Li. High-dynamic-range 3D shape measurement utilizing the transitioning state of digital micromirror device. Opt. Lasers Eng., 107, 176-181(2018).

    [55] H. Jiang, H. Zhao, X. Li. High dynamic range fringe acquisition: a novel 3-D scanning technique for high-reflective surfaces. Opt. Lasers Eng., 50, 1484-1493(2012).

    [56] L. Zhang, Q. Chen, C. Zuo, S. Feng. Real-time high dynamic range 3D measurement using fringe projection. Opt. Express, 28, 24363-24378(2020).

    [57] L. Zhang, Q. Chen, C. Zuo, S. Feng. High-speed high dynamic range 3D shape measurement based on deep learning. Opt. Lasers Eng., 134, 106245(2020).

    [58] J. H. Bruning, D. R. Herriott, J. Gallagher, D. Rosenfeld, A. White, D. Brangaccio. Digital wavefront measuring interferometer for testing optical surfaces and lenses. Appl. Opt., 13, 2693-2703(1974).

    [59] Y. Yin, Z. Cai, H. Jiang, X. Meng, J. Xi, X. Peng. High dynamic range imaging for fringe projection profilometry with single-shot raw data of the color camera. Opt. Lasers Eng., 89, 138-144(2017).

    [60] M. Wang, G. Du, C. Zhou, C. Zhang, S. Si, H. Li, Z. Lei, Y. Li. Enhanced high dynamic range 3D shape measurement based on generalized phase-shifting algorithm. Opt. Commun., 385, 43-53(2017).

    [61] Y. Chen, Y. He, E. Hu. Phase deviation analysis and phase retrieval for partial intensity saturation in phase-shifting projected fringe profilometry. Opt. Commun., 281, 3087-3090(2008).

    [62] Z. Qi, Z. Wang, J. Huang, C. Xing, J. Gao. Error of image saturation in the structured-light method. Appl. Opt., 57, A181-A188(2018).

    [63] P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, P.-A. Manzagol, L. Bottou. Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res., 11, 3371-3408(2010).

    [64] D. J. Im, S. Ahn, R. Memisevic, Y. Bengio. Denoising criterion for variational auto-encoding framework. AAAI Conference on Artificial Intelligence, 2059-2062(2017).

    [65] Y. Kiarashinejad, M. Zandehshahvar, S. Abdollahramezani, O. Hemmatyar, R. Pourabolghasem, A. Adibi. Knowledge discovery in nanophotonics using geometric deep learning. Adv. Intell. Syst., 2, 1900132(2020).

    [66] Y. Rivenson, Z. Göröcs, H. Günaydin, Y. Zhang, H. Wang, A. Ozcan. Deep learning microscopy. Optica, 4, 1437-1443(2017).

    [67] Y. Rivenson, Y. Zhang, H. Günaydn, D. Teng, A. Ozcan. Phase recovery and holographic image reconstruction using deep learning in neural networks. Light Sci. Appl., 7, 17141(2018).

    [68] M. Lyu, W. Wang, H. Wang, H. Wang, G. Li, N. Chen, G. Situ. Deep-learning-based ghost imaging. Sci. Rep., 7, 17865(2017).

    [69] Y. Li, Y. Xue, L. Tian. Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media. Optica, 5, 1181-1190(2018).

    [70] C. S. Lee, A. J. Tyring, N. P. Deruyter, Y. Wu, A. Rokem, A. Y. Lee. Deep-learning based, automated segmentation of macular edema in optical coherence tomography. Biomed. Opt. Express, 8, 3440-3448(2017).

    [71] S. Feng, Q. Chen, G. Gu, T. Tao, L. Zhang, Y. Hu, W. Yin, C. Zuo. Fringe pattern analysis using deep learning. Adv. Photonics, 1, 025001(2019).

    [72] J. Qian, S. Feng, Y. Li, T. Tao, J. Han, Q. Chen, C. Zuo. Single-shot absolute 3D shape measurement with deep-learning-based color fringe projection profilometry. Opt. Lett., 45, 1842-1845(2020).

    [73] J. Shi, X. Zhu, H. Wang, L. Song, Q. Guo. Label enhanced and patch based deep learning for phase retrieval from single frame fringe pattern in fringe projection 3D measurement. Opt. Express, 27, 28929-28943(2019).

    [74] T. Yang, Z. Zhang, H. Li, X. Li, X. Zhou. Single-shot phase extraction for fringe projection profilometry using deep convolutional generative adversarial network. Meas. Sci. Technol., 32, 015007(2020).

    [75] W. Yin, Q. Chen, S. Feng, T. Tao, L. Huang, M. Trusiak, A. Asundi, C. Zuo. Temporal phase unwrapping using deep learning. Sci. Rep., 9, 20175(2019).

    [76] J. Qian, S. Feng, T. Tao, Y. Hu, Y. Li, Q. Chen, C. Zuo. Deep-learning-enabled geometric constraints and phase unwrapping for single-shot absolute 3D shape measurement. APL Photonics, 5, 046105(2020).

    [77] O. Ronneberger, P. Fischer, T. Brox. U-net: convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, 234-241(2015).

    [78] C. Zuo, T. Tao, S. Feng, L. Huang, A. Asundi, Q. Chen. Micro Fourier transform profilometry (μftp): 3D shape measurement at 10,000 frames per second. Opt. Lasers Eng., 102, 70-91(2018).

    CLP Journals

    [1] Zheng Sun, Minghui Duan, Yabing Zheng, Yi Jin, Xin Fan, Jinjin Zheng, "Intensity diffusion: a concealed cause of fringe distortion in fringe projection profilometry," Photonics Res. 10, 1210 (2022)

    Tools

    Get Citation

    Copy Citation Text

    Shijie Feng, Chao Zuo, Liang Zhang, Wei Yin, Qian Chen, "Generalized framework for non-sinusoidal fringe analysis using deep learning," Photonics Res. 9, 1084 (2021)

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Image Processing and Image Analysis

    Received: Feb. 2, 2021

    Accepted: Apr. 13, 2021

    Published Online: May. 27, 2021

    The Author Email: Shijie Feng (shijiefeng@njust.edu.cn), Chao Zuo (zuochao@njust.edu.cn), Qian Chen (chenqian@njust.edu.cn)

    DOI:10.1364/PRJ.420944

    Topics