Chinese Journal of Lasers, Volume. 48, Issue 12, 1209001(2021)

3D Imaging Using Geometric Light Field: A Review

Yongkai Yin1、*, Kai Yu1, Chunzhan Yu1, Xuechun Bai1, Zewei Cai2, Xiangfeng Meng1, and Xiulun Yang1
Author Affiliations
  • 1School of Information Science and Engineering, Shandong University, Qingdao, Shandong 266237, China
  • 2School of Electronic and Optical Engineering, Nanjing University of Science and Technology, Nanjing, Jiangsu 210094, China
  • show less
    References(119)

    [3] Mendlovic D. Toward a super imaging system[J]. Applied Optics, 52, 561-566(2013).

    [4] Mait J N, Euliss G W, Athale R A. Computational imaging[J]. Advances in Optics and Photonics, 10, 409-483(2018).

    [11] Zhu H, Wang Q, Yu J Y. Light field imaging: models, calibrations, reconstructions, and applications[J]. Frontiers of Information Technology & Electronic Engineering, 18, 1236-1249(2017).

    [14] Marrugo A G, Gao F, Zhang S. State-of-the-art active optical techniques for three-dimensional surface metrology: a review[J]. Journal of the Optical Society of America A, 37, B60-B77(2020).

    [15] Martínez-Corral M, Javidi B. Fundamentals of 3D imaging and displays: a tutorial on integral imaging, light-field, and plenoptic systems[J]. Advances in Optics and Photonics, 10, 512-566(2018).

    [16] Chen N, Zuo C, Lam E Y et al. 3D imaging based on depth measurement technologies[J]. Sensors, 18, 3711-3749(2018).

    [17] Nie Y F, Xiangli B, Zhou Z L. Advances in light field photography technique[J]. Journal of the Graduate School of the Chinese Academy of Sciences, 28, 563-572(2011).

    [18] Zhang C, Liu F, Hou G Q et al. Light field photography and its application in computer vision[J]. Journal of Image and Graphics, 21, 263-281(2016).

    [20] Gershun A. The light field[J]. Journal of Mathematics and Physics, 18, 51-151(1939).

    [21] Adelson E H, Bergen J R . The plenoptic function and the elements of early vision[M]. //Landy M, Movshon J A. Computational models of visual processing, 3-20(1991).

    [22] McMillan L, Bishop G. Plenoptic modeling: an image-based rendering system[C]. //Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques-SIGGRAPH’95, September 15, 1995, New York, NY, United States, 39-46(1995).

    [23] Levoy M, Hanrahan P. Light field rendering[C]. //Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques-SIGGRAPH’96, August 1, 1996, New York, NY, United States., 31-42(1996).

    [24] Stern A, Javidi B. Ray phase space approach for 3-D imaging and 3-D optical data representation[J]. Journal of Display Technology, 1, 141-150(2005).

    [25] Ng R. Digital light field photography[D](2006).

    [26] Hecht E. Optics[M]. 5th ed(2016).

    [27] Hartley R, Zisserman A. Multiple view geometry in computer vision[M](2004).

    [31] Yang J C, Everett M, Buehler C et al. A real-time distributed light field camera[M]. //Debevec P, Gibson S. Eurographics workshop on rendering, 77-86(2002).

    [32] Wilburn B, Joshi N, Vaish V et al. High performance imaging using large camera arrays[J]. ACM Transactions on Graphics (TOG), 24, 765-776(2005).

    [33] Guo K W, Lincoln P, Davidson P et al. The relightables[J]. ACM Transactions on Graphics (TOG), 38, 1-19(2019).

    [34] Lin X, Wu J, Zheng G et al. Camera array based light field microscopy[J]. Biomedical Optics Express, 6, 3179-3189(2015).

    [35] Venkataraman K, Lelescu D, Duparré J et al. PiCam: an ultra-thin high performance monolithic camera array[J]. ACM Transactions on Graphics, 32, 1-13(2013).

    [36] Lanman D, Crispell D, Wachs M et al. Spherical catadioptric arrays: construction, multi-view geometry, and calibration[C]. //Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’06), June 14-16, 2006, Chapel Hill, NC, USA, 81-88(2006).

    [37] Levoy M, Chen B, Vaish V et al. Synthetic aperture confocal imaging[J]. ACM Transactions on Graphics (TOG), 23, 825-834(2004).

    [38] Taguchi Y, Agrawal A, Veeraraghavan A et al. Axial-cones: modeling spherical catadioptric cameras for wide-angle light field rendering[C]. //ACM SIGGRAPH Asia 2010 Papers on-SIGGRAPH ASIA’10, December 15-18, 2010, Seoul, South Korea, 1-8(2010).

    [39] Georgiev T, Zheng K C, Curless B et al. Spatio-angular resolution tradeoffs in integral photography[M]. //Akenine-Moller T, Heidrich W. Symposium on rendering, 263-272(2006).

    [41] Ng R, Levoy M, Brédif M et al. Light field photography with a hand-held plenopic camera[R]. Stanford: Stanford Technical Report CTSR(2005).

    [42] Lumsdaine A, Georgiev T. The focused plenoptic camera[C]. //2009 IEEE International Conference on Computational Photography (ICCP), April 16-17, 2009, San Francisco, CA, USA., 1-8(2009).

    [43] Georgiev T, Lumsdaine A. The multifocus plenoptic camera[J]. Proceedings of SPIE, 8299, 829908(2012).

    [44] Isaksen A, McMillan L, Gortler S J. Dynamically reparameterized light fields[C]. //Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques - SIGGRAPH’00, July 1, 2000, New York, NY, United States., 297-306(2000).

    [45] Ng R. Fourier slice photography[C]. //ACM SIGGRAPH 2005 Papers on - SIGGRAPH ’05, July 31-August 4, 2005, Los Angeles, California., 735-744(2005).

    [47] Pujades S, Devernay F, Goldluecke B. Bayesian view synthesis and image-based rendering principles[C]. //2014 IEEE Conference on Computer Vision and Pattern Recognition, June 23-28, 2014, Columbus, OH, USA., 3906-3913(2014).

    [48] Kalantari N K, Wang T C, Ramamoorthi R. Learning-based view synthesis for light field cameras[J]. ACM Transactions on Graphics (TOG), 35, 1-10(2016).

    [49] Mildenhall B, Srinivasan P P, Ortiz-Cayon R et al. Local light field fusion[J]. ACM Transactions on Graphics, 38, 1-14(2019).

    [50] Johannsen O, Honauer K, Goldluecke B et al. A taxonomy and evaluation of dense light field depth estimation algorithms[C]. //2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), July 21-26, 2017, Honolulu, HI, USA., 1795-1812(2017).

    [51] Vaish V, Wilburn B, Joshi N et al. Using plane + parallax for calibrating dense camera arrays[C]. //Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004, June 27-July 2, 2004, Washington, DC, USA., 2-9(2004).

    [52] Seitz S M, Curless B, Diebel J et al. A comparison and evaluation of multi-view stereo reconstruction algorithms[C]. //2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), June 17-22, 2006, New York, NY, USA., 519-528(2006).

    [53] Yu Z, Guo X Q, Ling H B et al. Line assisted light field triangulation and stereo matching[C]. //2013 IEEE International Conference on Computer Vision, December 1-8, 2013, Sydney, NSW, Australia., 2792-2799(2013).

    [54] Heber S, Pock T. Shape from light field meets robust PCA[C]. //Fleet D, Pajdla T, Schiele B, et al. Computer Vision-ECCV 2014. Lecture notes in computer science, 8694, 751-767(2014).

    [55] Jeon H G, Park J, Choe G et al. Accurate depth map estimation from a lenslet light field camera[C]. //2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 7-12, 2015, Boston, MA, USA, 1547-1555(2015).

    [56] Jeon H G, Park J, Choe G et al. Depth from a light field image with learning-based matching costs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41, 297-310(2019).

    [58] Tao M W, Srinivasan P P, Hadap S et al. Shape estimation from shading, defocus, and correspondence using light-field angular coherence[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39, 546-560(2017).

    [59] Tao M W, Hadap S, Malik J et al. Depth from combining defocus and correspondence using light-field cameras[C]. //2013 IEEE International Conference on Computer Vision, December 1-8, 2013, Sydney, NSW, Australia, 673-680(2013).

    [60] Tao M W, Srinivasan P P, Malik J et al. Depth from shading, defocus, and correspondence using light-field angular coherence[C]. //2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 7-12, 2015, Boston, MA, USA., 1940-1948(2015).

    [61] Lin H T, Chen C, Kang S B et al. Depth recovery from light field using focal stack symmetry[C]. //2015 IEEE International Conference on Computer Vision (ICCV), December 7-13, 2015, Santiago, Chile., 3451-3459(2015).

    [63] Tian J D, Murez Z, Cui T et al. Depth and image restoration from light field in a scattering medium[C]. //2017 IEEE International Conference on Computer Vision (ICCV), October 22-29, 2017, Venice, Italy, 2401-2410(2017).

    [65] Cai Z W, Pedrini G, Osten W et al. Single-shot structured-light-field three-dimensional imaging[J]. Optics Letters, 45, 3256-3259(2020).

    [66] Cai Z, Liu X, Pedrini G et al. Light-field depth estimation considering plenoptic imaging distortion[J]. Optics Express, 28, 4156-4168(2020).

    [67] Chen C, Lin H T, Yu Z et al. Light field stereo matching using bilateral statistics of surface cameras[C]. //2014 IEEE Conference on Computer Vision and Pattern Recognition, June 23-28, 2014, Columbus, OH, USA., 1518-1525(2014).

    [68] Wang T C, Efros A A, Ramamoorthi R. Depth estimation with occlusion modeling using light-field cameras[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38, 2170-2181(2016).

    [71] Tao M W, Su J C, Wang T C et al. Depth estimation and specular removal for glossy surfaces using point and line consistency with light-field cameras[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38, 1155-1169(2016).

    [75] Wanner S, Fehr J, Jähne B. Generating EPI representations of 4D light fields with a single lens focused plenoptic camera[M]. //Bebis G, Boyle R, Parvin B, et al. Advances in visual computing. Lecture notes in computer science, 6938, 90-101(2011).

    [76] Wanner S, Goldluecke B. Globally consistent depth labeling of 4D light fields[C]. //2012 IEEE Conference on Computer Vision and Pattern Recognition, June 16-21, 2012, Providence, RI, USA., 41-48(2012).

    [77] Tosic I, Berkner K. Light field scale-depth space transform for dense depth estimation[C]. //2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops, June 23-28, 2014, Columbus, OH, USA., 441-448(2014).

    [78] Li J Q, Lu M L, Li Z N. Continuous depth map reconstruction from light fields[J]. IEEE Transactions on Image Processing, 24, 3257-3265(2015).

    [79] Kim C, Zimmer H, Pritch Y et al. Scene reconstruction from high spatio-angular resolution light fields[J]. ACM Transactions on Graphics (TOG), 32, 1-12(2013).

    [80] Zhang S, Sheng H, Li C et al. Robust depth estimation for light field via spinning parallelogram operator[J]. Computer Vision and Image Understanding, 145, 148-159(2016).

    [81] Cai Z W, Liu X L, Pedrini G et al. Structured-light-field 3D imaging without phase unwrapping[J]. Optics and Lasers in Engineering, 129, 106047(2020).

    [82] Liu L, Xiang S, Deng H P et al. Fast geometry estimation for phase-coding structured light field[C]. //2020 IEEE International Conference on Visual Communications and Image Processing (VCIP), December 1-4, 2020, Macao, China., 124-127(2020).

    [83] Heber S, Pock T. Convolutional networks for shape from light field[C]. //2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 27-30, 2016, Las Vegas, NV, USA., 3746-3754(2016).

    [84] Heber S, Yu W, Pock T. Neural EPI-volume networks for shape from light field[C]. //2017 IEEE International Conference on Computer Vision (ICCV), October 22-29, 2017, Venice, Italy., 2271-2279(2017).

    [85] Wu G C, Zhao M D, Wang L Y et al. Light field reconstruction using deep convolutional network on EPI[C]. //2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 21-26, 2017, Honolulu, HI, USA., 1638-1646(2017).

    [86] Shin C, Jeon H G, Yoon Y et al. EPINET: a fully-convolutional neural network using epipolar geometry for depth from light field images[C]. //2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 18-23, 2018, Salt Lake City, UT, USA, 4748-4757(2018).

    [87] Wang X, Tao C, Wu R et al. Light-field-depth-estimation network based on epipolar geometry and image segmentation[J]. Journal of the Optical Society of America A, Optics, Image Science, and Vision, 37, 1236-1243(2020).

    [88] Johannsen O, Sulc A, Goldluecke B. What sparse light field coding reveals about scene structure[C]. //2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 27-30, 2016, Las Vegas, NV, USA., 3262-3270(2016).

    [89] Hartley R I, Sturm P. Triangulation[J]. Computer Vision and Image Understanding, 68, 146-157(1997).

    [90] Johannsen O, Sulc A, Goldluecke B. On linear structure from motion for light field cameras[C]. //2015 IEEE International Conference on Computer Vision (ICCV), December 7-13, 2015, Santiago, Chile., 720-728(2015).

    [91] Zhang Y L, Li Z, Yang W et al. The light field 3D scanner[C]. //2017 IEEE International Conference on Computational Photography (ICCP), May 12-14, 2017, Stanford, CA, USA., 1-9(2017).

    [92] Zhang Y L, Yu P H, Yang W et al. Ray space features for plenoptic structure-from-motion[C]. //2017 IEEE International Conference on Computer Vision (ICCV), October 22-29, 2017, Venice, Italy., 4641-4649(2017).

    [93] Zeller N, Quint F, Stilla U. From the calibration of a light-field camera to direct plenoptic odometry[J]. IEEE Journal of Selected Topics in Signal Processing, 11, 1004-1019(2017).

    [94] Fahringer T W, Lynch K P, Thurow B S. Volumetric particle image velocimetry with a single plenoptic camera[J]. Measurement Science and Technology, 26, 115201(2015).

    [96] Shi S X, Ding J F, New T H et al. Light-field camera-based 3D volumetric particle image velocimetry with dense ray tracing reconstruction technique[J]. Experiments in Fluids, 58, 1-16(2017).

    [98] Sun J, Hossain M M, Xu C L et al. A novel calibration method of focused light field camera for 3-D reconstruction of flame temperature[J]. Optics Communications, 390, 7-15(2017).

    [99] Dansereau D G, Pizarro O, Williams S B. Decoding, calibration and rectification for lenselet-based plenoptic cameras[C]. //2013 IEEE Conference on Computer Vision and Pattern Recognition, June 23-28, 2013, Portland, OR, USA., 1027-1034(2013).

    [100] Monteiro N B, Marto S, Barreto J P et al. Depth range accuracy for plenoptic cameras[J]. Computer Vision and Image Understanding, 168, 104-117(2018).

    [103] Hahne C, Aggoun A, Velisavljevic V et al. Baseline and triangulation geometry in a standard plenoptic camera[J]. International Journal of Computer Vision, 126, 21-35(2018).

    [104] Bok Y, Jeon H, Kweon I S. Geometric calibration of micro-lens-based light field cameras using line features[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39, 287-300(2017).

    [106] Hall E M, Fahringer T W, Guildenbecher D R et al. Volumetric calibration of a plenoptic camera[J]. Applied Optics, 57, 914-923(2018).

    [108] Cai Z W, Liu X L, Chen Z Z et al. Light-field-based absolute phase unwrapping[J]. Optics Letters, 43, 5717-5720(2018).

    [110] Cai Z W, Liu X L, Tang Q J et al. Light field 3D measurement using unfocused plenoptic cameras[J]. Optics Letters, 43, 3746-3749(2018).

    [111] Cai Z W, Liu X L, Pedrini G et al. Unfocused plenoptic metric modeling and calibration[J]. Optics Express, 27, 20177-20198(2019).

    [113] Lin R J, Su V C, Wang S M et al. Achromatic metalens array for full-colour light-field imaging[J]. Nature Nanotechnology, 14, 227-231(2019).

    [114] Fan Z B, Qiu H Y, Zhang H L et al. A broadband achromatic metalens array for integral imaging in the visible[J]. Light: Science & Applications, 8, 67(2019).

    [115] Orth A, Ploschner M, Wilson E R et al. Optical fiber bundles: ultra-slim light field imaging probes[J]. Science Advances, 5, eaav1555(2019).

    [119] Liu J D, Zaouter C, Liu X L et al. Coded-aperture broadband light field imaging using digital micromirror devices[J]. Optica, 8, 139-142(2021).

    Tools

    Get Citation

    Copy Citation Text

    Yongkai Yin, Kai Yu, Chunzhan Yu, Xuechun Bai, Zewei Cai, Xiangfeng Meng, Xiulun Yang. 3D Imaging Using Geometric Light Field: A Review[J]. Chinese Journal of Lasers, 2021, 48(12): 1209001

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: holography and information processing

    Received: Mar. 1, 2021

    Accepted: Apr. 27, 2021

    Published Online: Jun. 7, 2021

    The Author Email: Yin Yongkai (yinyongkai@sdu.edu.cn)

    DOI:10.3788/CJL202148.1209001

    Topics