Infrared and Laser Engineering, Volume. 51, Issue 1, 20210935(2022)

Progress and challenges in dynamic holographic 3D display for the metaverse (Invited)

Liangcai Cao, Zehao He*, Kexuan Liu, and Xiaomeng Sui
Author Affiliations
  • State Key Laboratory of Precision Measurement Technology and Instrument, Department of Precision Instrument, Tsinghua University, Beijing 100084, China
  • show less
    References(121)

    [1] [1] Marieb E N, Hoehn K N. Human Anatomy & Physiology [M]. 11th ed. London: Pearson, 2018.

    [2] Geng J. Three-dimensional display technologies[J]. Advances in Optics and Photonics, 5, 456-535(2013).

    [3] [3] 5G unlocks a wld of opptunities [ROL]. [20220114].https:www.huawei.comustechnologyinsightsindustryinsightsoutlookmobilebroadbinsightsrepts5 gunlocksawldofopptunities.

    [4] [4] Gr View Research. 3D display market size & share trends analysis rept [DBOL].[20180207]. https:www.grviewresearch.comindustryanalysis3 ddisplaymarket.

    [5] [5] Blundell B, Schwarz A. Volumetric Three Dimensional Display System [M]. New Yk: WileyIEEE Press, 1999.

    [6] [6] Okoshi T. Threedimensional Imaging Techniques [M]. New Yk: Academic Press, 1976.

    [7] Xu B, Wu Q, Bao Y, et al. Time-multiplexed stereoscopic display with a quantum dot-polymer scanning backlight[J]. Applied Optics, 58, 4526-4532(2019).

    [8] North T, Wagner M, Bourquin S, et al. Compact and high-brightness helmet-mounted head-up display system by retinal laser projection[J]. Journal of Display Technology, 12, 982-985(2016).

    [9] Wang Y, Liu W, Meng X, et al. Development of an immersive virtual reality head-mounted display with high performance[J]. Applied Optics, 55, 6969-6977(2016).

    [10] Hiura H, Komine K, Arai J, et al. Measurement of static convergence and accommodation responses to images of integral photography and binocular stereoscopy[J]. Optics Express, 25, 3454-3468(2017).

    [11] Zhuang Z, Zhang L, Surman P, et al. Addressable spatial light modulators for eye-tracking autostereoscopic three-dimensional display using a scanning laser[J]. Applied Optics, 57, 4457-4466(2018).

    [12] Meng Y, Lyu Y, Chen L L, et al. Motion parallax and lossless resolution autostereoscopic 3 D display based on a binocular viewpoint tracking liquid crystal dynamic grating adaptive screen[J]. Optics Express, 29, 35456-35473(2021).

    [13] Lin Y, Liu X, Liu X, et al. Three-dimensional volumetric display system utilizing a rotating two-dimensional LED array[J]. Acta Optica Sinica, 23, 1158-1162(2003).

    [14] Lu H, Zhang J, Song Z, et al. Submillisecond-response light shutter for solid-state volumetric 3 D display based on polymer-stabilized cholesteric texture[J]. Journal of Display Technology, 19, 396-400(2014).

    [15] Gong D, Wang C, Wang X, et al. Static volumetric three-dimensional display based on an electric-field-controlled two-dimensional optical beam scanner[J]. Applied Optics, 58, 7067-7072(2019).

    [16] Kumagai K, Yamaguchi I, Hayasaki Y. Three-dimensionally structured voxels for volumetric display[J]. Optics Letters, 43, 3341-3344(2018).

    [17] Tian F, Wang H, Fang Y, et al. A swept volume display system using a planetary gear structure based on parallel moving[J]. Journal of Display Technology, 8, 457-463(2012).

    [18] Xie W, Wang Y, Deng H, et al. Viewing angle-enhanced integral imaging system using three lens arrays[J]. Chinese Optics Letters, 12, 011101(2014).

    [19] Ren H, Xing Y, Zhang H L, et al. 2D/3D mixed display based on integral imaging and a switchable diffuser element[J]. Applied Optics, 58, G276-G281(2019).

    [20] Zhang H -L, Deng H, Ren H, et al. Method to eliminate pseudoscopic issue in an integral imaging 3 D display by using a transmissive mirror device and light filter[J]. Optics Letters, 45, 351-354(2020).

    [21] Yang L, Sang X, Yu X, et al. A crosstalk-suppressed dense multi-view light-field display based on real-time light-field pickup and reconstruction[J]. Optics Express, 26, 34412-34427(2018).

    [22] Chen D, Sang X, Yu X, et al. Performance improvement of compressive light field display with the viewing-position-dependent weight distribution[J]. Optics Express, 24, 29781-29793(2016).

    [23] [23] Adelson E H, Bergen J R. The Plenoptic Function the Elements of Early Vision [M] Lry M. Movshon J A. Computational models of Visual Processing. Cambridge: MIT Press, 1991: 320.

    [24] [24] Wenger A, Gardner A, Tchou C, et al. Perfmance relighting reflectance transfmation with timemultiplexed illumination [C]Special Interest Group on Computer Graphics Interactive Techniques Conference (SIGGRAPH), 2005, 24: 756764.

    [25] Ma Q, Cao L, He Z, et al. Progress of three-dimensional light-field display[J]. Chinese Optics Letters, 17, 111001(2019).

    [26] Huang H, Hua H. Systematic characterization and optimization of 3D light field displays[J]. Optics Express, 25, 18508-18525(2017).

    [27] Xu M, Hua H. Systematic method for modeling and characterizing multilayer light field displays[J]. Optics Express, 28, 1014-1036(2020).

    [28] [28] Goodman J P. Introduction to Fourier Optics [M]. 4th ed. New Yk: W. H. Freeman & Company, 2017.

    [29] Pi D, Liu J, Kang R, et al. Reducing the memory usage of computer-generated hologram calculation using accurate high-compressed look-up-table method in color 3D holographic display[J]. Optics Express, 27, 28410-28422(2019).

    [30] Wang Z, Lv G Q, Feng Q B, et al. Resolution priority holographic stereogram based on integral imaging with enhanced depth range[J]. Optics Express, 27, 2689-2702(2019).

    [31] Wang Z, Zhu L M, Zhang X, et al. Computer-generated photorealistic hologram using ray-wavefront conversion based on the additive compressive light field approach[J]. Optics Letters, 45, 615-618(2020).

    [32] Chang C, Cui W, Gao L. Holographic multiplane near-eye display based on amplitude-only wavefront modulation[J]. Optics Express, 27, 30960-30970(2019).

    [33] Sui X, He Z, Jin G, et al. Band-limited double-phase method for enhancing image sharpness in complex modulated computer-generated holograms[J]. Optics Express, 29, 2597-2612(2021).

    [34] Sui X, He Z, Zhang H, et al. Spatiotemporal double-phase hologram for complex-amplitude holographic displays[J]. Chinese Optics Letters, 18, 100901(2020).

    [35] Liu K, He Z, Cao L. Pattern-adaptive error diffusion algorithm for improved phase-only hologram generation[J]. Chinese Optics Letters, 19, 050501(2021).

    [36] Li C, Cao L, Wang Z, et al. Hybrid polarization-angle multiplexing for volume holography in gold nanoparticle-doped photopolymer[J]. Optics Letters, 39, 6891-6894(2014).

    [37] Lee J S, Kim Y K, et al. See-through display combined with holographic display and Maxwellian display using switchable holographic optical element based on liquid lens[J]. Optics Express, 26, 19341-19355(2018).

    [38] Lohmann A W. On Moiré fringes as Fourier test objects[J]. Applied Optics, 5, 669-670(1966).

    [39] Brown B R, Lohmann A W. Complex spatial filtering with binary masks[J]. Applied Optics, 5, 967-969(1966).

    [40] Waters J P. Holographic image synthesis utilizing theoretical methods[J]. Applied Physics Letters, 9, 405-407(1967).

    [41] Lesem L B, Hirsch P M, Jordan J A. The kinoform: a new wavefront reconstruction device[J]. IBM Journal of Research and Development, 13, 150-155(1969).

    [42] Lee W H. Sampled Fourier transform hologram generated by computer[J]. Applied Optics, 9, 639-643(1970).

    [43] Leseberg D, Frère C. Computer-generated holograms of 3D objects composed of tilted planar segments[J]. Applied Optics, 27, 3020-3024(1988).

    [44] Yamaguchi M, Endoh H, Honda T, et al. High-quality recording of a full-parallax holographic stereogram with a digital diffuser[J]. Optics Letters, 19, 135-137(1994).

    [45] [45] Yoshikawa H, Kameyama H. Integral holography [C]Proceeding of SPIE, 1995, 2406: 226234.

    [46] [46] Benton S A. Synthetic holography [C]Conference on Lasers ElectroOptics, 1989, 11: JA1.

    [47] Hilaire P S, Benton S A, Lucente M. Synthetic aperture holography: a novel approach to three-dimensional displays[J]. Journal of the Optical Society of America A, 9, 1969-1977(1992).

    [48] Lucente M. Interactive computation of holograms using a look-up table[J]. Journal of Electronic Imaging, 2, 28-34(1993).

    [49] Nishi S, Shiba K, Mori K, et al. Fast calculation of computer-generated Fresnel hologram utilizing distributed parallel processing and array operation[J]. Optical Review, 12, 287-292(2005).

    [50] Ahrenberg L, Benzie P, Magnor M, et al. Computer generated holography using parallel commodity graphics hardware[J]. Optics Express, 14, 7636-7641(2006).

    [51] Huebschman M L, Munjuluri B, Garner H R. Dynamic holographic 3-D image projection[J]. Optics Express, 11, 437-445(2003).

    [52] Guo C S, Rong Z Y, Wang H T, et al. Phase-shifting with computer-generated holograms written on a spatial light modulator[J]. Applied Optics, 42, 6975-6979(2003).

    [53] Matsushima K, Schimmel H, Wyrowski F. Fast calculation method for optical diffraction on tilted planes by use of the angular spectrum of plane waves[J]. Journal of the Optical Society of America A, 20, 1755-1762(2003).

    [54] Matsushima K, Nakahara S. Extremely high-definition full-parallax computer-generated hologram created by the polygon-based method[J]. Applied Optics, 48, H54-H63(2003).

    [55] Ahrenberg L, Benzie P, Magnor M, et al. Computer generated holograms from three dimensional meshes using an analytic light transport model[J]. Applied Optics, 47, 1567-1574(2008).

    [56] Wakunami K, Yamaguchi M. Calculation for computer generated hologram using ray-sampling plane[J]. Optics Express, 19, 9086-9101(2011).

    [57] Kurihara T, Takaki Y. Shading of a computer-generated hologram by zone plate modulation[J]. Optics Express, 20, 3529-3540(2012).

    [58] Ichikawa T, Yamaguchi K, Sakamoto Y. Realistic expression for full-parallax computer-generated holograms with the ray-tracing method[J]. Applied Optics, 52, A201-A209(2013).

    [59] Zhao Y, Cao L, Zhang H, et al. Accurate calculation of computer-generated holograms using angular-spectrum layer-oriented method[J]. Optics Express, 23, 25440-25449(2015).

    [60] Smalley D E, Smithwick Q Y J, Bove V M, et al. Anisotropic leaky-mode modulator for holographic video displays[J]. Nature, 498, 313-317(2013).

    [61] Inoue T, Takaki Y. Table screen 360-degree holographic display using circular viewing-zone scanning[J]. Optics Express, 23, 6533-6542(2015).

    [62] Horisaki R, Takagi R, Tanida J. Deep-learning-generated holography[J]. Applied Optics, 57, 3859-3863(2018).

    [63] Blinder D. Direct calculation of computer-generated holograms in sparse bases[J]. Optics Express, 27, 23124-23137(2019).

    [64] Cencillo-Abad P, Plum E, Rogers E T F, et al. Spatial optical phase-modulating metadevice with subwavelength pixelation[J]. Optics Express, 24, 18790-18798(2016).

    [65] Martins A, Li J, da Mota A F, et al. Broadband C-Si metasurfaces with polarization control at visible wavelengths: applications to 3 D stereoscopic holography[J]. Optics Express, 26, 30740-30752(2018).

    [66] Wu J, Fu S, Zhang X, et al. Graphene-oxide/TiO2 nanocomposite films with electron-donors for multicolor holography[J]. Optics Express, 27, 1740-1749(2019).

    [67] Jiang Q, Cao L, Zhang H, et al. Improve the quality of holographic image with complex-amplitude metasurface[J]. Optics Express, 27, 33700-33708(2019).

    [68] Li J, Smithwick Q, Chu D. Scalable coarse integral holographic video display with integrated spatial image tiling[J]. Optics Express, 28, 9899-9912(2020).

    [69] An J, Won K, Kim Y, et al. Slim-panel holographic video display[J]. Nature Communications, 11, 5568(2020).

    [70] Shi L, Li B, Kim C, et al. Towards real-time photorealistic 3D holography with deep neural networks[J]. Nature, 591, 234-239(2021).

    [71] Lohmann A W, Paris D P. Binary Fraunhofer holograms, generated by computer[J]. Applied Optics, 6, 1739-1748(1967).

    [72] Shimobaba T, Ito T. Random phase-free computer-generated hologram[J]. Optics Express, 23, 9549-9554(2015).

    [73] Wyrowski F, Bryngdahl O. Speckle-free reconstruction in digital holography[J]. Journal of the Optical Society of America A, 6, 1171-1174(1989).

    [74] Zea A V, Torroba R. Optimized random phase tiles for non-iterative hologram generation[J]. Applied Optics, 58, 9013-9019(2019).

    [75] Ma H, Liu J, Yang M, et al. Influence of limited random-phase of objects on the image quality of 3 D holographic display[J]. Optics Communications, 385, 153-159(2017).

    [76] Zhao T, Liu J, Duan J, et al. Image quality enhancement via gradient-limited random phase addition in holographic display[J]. Optics Communications, 442, 84-89(2019).

    [77] Nagahama Y, Shimobaba T, Kakue T, et al. Image quality improvement of random phase-free holograms by addressing the cause of ringing artifacts[J]. Applied Optics, 58, 2146-2151(2019).

    [78] Mengu D, Ulusoy E, Urey H. Non-iterative phase hologram computation for low speckle holographic image projection[J]. Optics Express, 24, 4462-4476(2016).

    [79] Cruz M L. Full image reconstruction with reduced speckle noise, from a partially illuminated Fresnel hologram, using a structured random phase[J]. Applied Optics, 58, 1917-1923(2019).

    [80] He Z, Sui X, Zhang H, et al. Frequency-based optimized random phase for computer-generated holographic display[J]. Applied Optics, 60, A145-A154(2021).

    [81] Gerchberg R W, Saxton W O. A practical algorithm for the determination of phase from image and diffraction plane pictures[J]. Optik, 35, 1-6(1971).

    [82] Chen C Y, Deng Q L, Wu P J, et al. Speckle reduction by combination of digital filter and optical suppression in a modified Gerchberg-Saxton algorithm computer-generated hologram[J]. Applied Optics, 53, G163-G168(2014).

    [83] Deng Q L, Lin B S, Chang H T, et al. MGSA-type computer-generated holography for vision training with head-mounted display[J]. Journal of Display Technology, 10, 433-437(2014).

    [84] Chen C Y, Chang H T, Chang T J, et al. Full-color and less-speckled modified Gerchberg-Saxton algorithm computer-generated hologram floating in a dual-parabolic projection system[J]. Chinese Optics Letters, 13, 110901(2015).

    [85] Liu S C, Chu D. Deep learning for hologram generation[J]. Optics Express, 29, 27373-27395(2021).

    [86] Kang J W, Park B S, Kim J K, et al. Deep-learning-based hologram generation using a generative model[J]. Applied Optics, 60, 7391-7399(2021).

    [87] Goi H, Komuro K, Nomura T. Deep-learning-based binary hologram[J]. Applied Optics, 59, 7103-7108(2020).

    [88] Lee J, Jeong J, Cho J, et al. Deep neural network for multi-depth hologram generation and its training strategy[J]. Optics Express, 28, 27137-27154(2020).

    [89] Shimobaba T, Blinder D, Makowski M, et al. Dynamic-range compression scheme for digital hologram using a deep neural network[J]. Optics Letters, 44, 3038-3041(2019).

    [90] Wu J, Liu K, Sui X, et al. High-speed computer-generated holography using an autoencoder-based deep neural network[J]. Optics Letters, 46, 2908-2011(2021).

    [91] Yao K, Wang J, Liu X, et al. Analysis of a holographic laser adaptive optics system using a deformable mirror[J]. Applied Optics, 56, 6639-6648(2017).

    [92] Andersen G, Austin P G, Gaddipati R, et al. Fast, compact, autonomous holographic adaptive optics[J]. Optics Express, 22, 9432-9441(2014).

    [93] Neil M A, Booth M J, Wilson T. Closed-loop aberration correction by use of a modal Zernike wave-front sensor[J]. Optics Express, 25, 1083-1085(2000).

    [94] Otón J, Ambs P, Millán M S, et al. Multipoint phase calibration for improved compensation of inherent wavefront distortion in parallel aligned liquid crystal on silicon displays[J]. Applied Optics, 46, 5667-5679(2007).

    [95] Yeom H J, Kim H J, Kim S B, et al. 3 D holographic head mounted display using holographic optical elements with astigmatism aberration compensation[J]. Optics Express, 23, 32025-32034(2015).

    [96] Kaczorowski A, Gordon G S, Wilkinson T D. Adaptive, spatially-varying aberration correction for real-time holographic projectors[J]. Optics Express, 24, 15742-15756(2016).

    [97] Kaczorowski A, Gordon G S, Palani A, et al. Optimization-based adaptive optical correction for holographic projectors[J]. Journal of Display Technology, 11, 596-603(2015).

    [98] Haist T, Peter A, Osten W. Holographic projection with field-dependent aberration correction[J]. Optics Express, 23, 5590-5595(2015).

    [99] He Z, Sui X, Jin G, et al. Distortion-correction method based on angular spectrum algorithm for holographic display[J]. IEEE Transactions on Industrial Informatics, 15, 6162-6169(2019).

    [100] [100] Silva R. 3D TV is dead what you need to know [ROL].[20130117].https:www.lifewire.comwhy3 dtvdied4126776.

    [101] Chen R H Y, Wilkinson T D. Computer generated hologram from point cloud using graphics processor[J]. Applied Optics, 48, 6841-6850(2009).

    [102] Su P, Cao W, Ma J, et al. Fast computer-generated hologram generation method for three-dimensional point cloud model[J]. Journal of Display Technology, 12, 1688-1694(2016).

    [103] Chen J S, Chu D P. Improved layer-based method for rapid hologram generation and real-time interactive holographic display applications[J]. Optics Express, 23, 18143-18155(2015).

    [104] Jia J, Si J, Chu D. Fast two-step layer-based method for computer generated hologram using sub-sparse 2 D fast Fourier transform[J]. Optics Express, 26, 17487-17497(2018).

    [105] Pan Y, Wang Y, Liu J, et al. Fast polygon-based method for calculating computer-generated holograms in three-dimensional display[J]. Applied Optics, 52, A290-A299(2013).

    [106] Liu J P, Liao H K. Fast occlusion processing for a polygon-based computer-generated hologram using the slice-by-slice silhouette method[J]. Applied Optics, 57, A215-A221(2018).

    [107] Yamaguchi M, Wakunami K, Inaniwa M. Computer generated hologram from full-parallax 3 D image data captured by scanning vertical camera array[J]. Chinese Optics Letters, 12, 060018(2014).

    [108] Yanagihara H, Kakue T, Yamamoto Y, et al. Real-time three-dimensional video reconstruction of real scenes with deep depth using electro-holographic display system[J]. Optics Express, 27, 15662-15678(2019).

    [109] Ichihashi Y, Oi R, Senoh T, et al. Real-time capture and reconstruction system with multiple GPUs for a 3 D live scene by a generation from 4 K IP images to 8 K holograms[J]. Optics Express, 20, 21645-21655(2012).

    [110] Wu J, Chen H, Liu X, et al. Unsupervised texture reconstruction method using bidirectional similarity function for 3-D measurements[J]. Optics Communications, 439, 85-93(2019).

    [111] Yamaguchi M. Light-field and holographic three-dimensional displays[J]. Journal of Optics Society of America A, 33, 2348-2364(2016).

    [112] Igarashi S, Nakamura T, Matsushima K, et al. Efficient tiled calculation of over-10-gigapixel holograms using ray-wavefront conversion[J]. Optics Express, 26, 10773-10786(2018).

    [113] Tsai S F, Cheng C C, Li C T, et al. A real-time 1080 p 2 D-to-3 D video conversion system[J]. IEEE Transactions on Consumer Electronics, 57, 915-922(2011).

    [114] Cheng C C, Li C T, Chen L G. A novel 2D-to-3D conversion system using edge information[J]. IEEE Transactions on Consumer Electronics, 56, 1739-1745(2010).

    [115] Lai Y K, Lai Y F, Chen Y C. An effective hybrid depth-generation algorithm for 2D-to-3D conversion in 3 D displays[J]. Journal of Display Technology, 9, 154-161(2013).

    [116] Zhang Z, Yin S, Liu L, et al. A real-time time-consistent 2D-to-3D video conversion system using color histogram[J]. IEEE Transactions on Consumer Electronics, 61, 524-530(2015).

    [117] Gil J, Kim M. Motion depth generation using MHI for 2D-to-3D video conversion[J]. Electronics Letters, 53, 1520-1522(2017).

    [118] He Z, Sui X, Cao L. Holographic 3D display using depth maps generated by 2 D-to-3 D rendering approach[J]. Applied Sciences, 11, 9889(2021).

    [119] Huang W, Cao X, Lu K, et al. Toward naturalistic 2D-to-3D conversion[J]. IEEE Transactions on Image Processing, 24, 724-733(2015).

    [120] Konrad J, Wang M, Ishwar P, et al. Learning-based, automatic 2D-to-3D image and video conversion[J]. IEEE Transactions on Image Processing, 22, 3485-3496(2013).

    [121] J L Herrera, C R del-Blanco, N García. A novel 2D to 3D video conversion system based on a machine learning approach[J]. IEEE Transactions on Consumer Electronics, 62, 429-436(2016).

    CLP Journals

    [1] Chao Zuo, Qian Chen. Computational optical imaging: An overview[J]. Infrared and Laser Engineering, 2022, 51(2): 20220110

    [2] Zehao HE, Yunhui GAO, Liangcai CAO, Yan ZHANG. Deep learning empowers generation and presentation of virtual-real fusion scenarios in holographic metaverse: development and prospects (invited)[J]. Infrared and Laser Engineering, 2025, 54(7): 20250189

    [3] Fuguo LYU, Furong HUO, Changxi XUE. Design of harmonic diffraction structure in glasses-free three-dimensional display with high brightness and low chromatic[J]. Infrared and Laser Engineering, 2024, 53(6): 20240097

    Tools

    Get Citation

    Copy Citation Text

    Liangcai Cao, Zehao He, Kexuan Liu, Xiaomeng Sui. Progress and challenges in dynamic holographic 3D display for the metaverse (Invited)[J]. Infrared and Laser Engineering, 2022, 51(1): 20210935

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Optical imaging

    Received: Nov. 1, 2021

    Accepted: Dec. 28, 2021

    Published Online: Mar. 8, 2022

    The Author Email:

    DOI:10.3788/IRLA20210935

    Topics