Techniques on Deep Learning‐Based Computer‐Generated Hologram (Invited)

Chongli Zhong; Xinzhu Sang; Binbin Yan

doi:10.3788/AOS251197

Acta Optica Sinica, Volume. 45, Issue 17, 1720014(2025)

Techniques on Deep Learning‐Based Computer‐Generated Hologram (Invited)

Chongli Zhong, Xinzhu Sang^*, and Binbin Yan^**

State Key Laboratory of Information Photonics and Optical Communications, School of Electronic Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China

show less

Abstract Get PDF(in Chinese)

References(100)

[1] Zhang H J[M]. Vision and its application technology(2004).

[2] Blinder D, Birnbaum T, Ito T et al. The state-of-the-art in computer generated holography for 3D display[J]. Light: Advanced Manufacturing, 3, 572-600(2022).

[3] Liang H, Wong W C, An T L et al. Metasurface-enabled quantum holograms with hybrid entanglement[J]. Advanced Photonics, 7, 026006(2025).

[4] Wen D D, Cadusch J J, Meng J J et al. Light field on a chip: metasurface-based multicolor holograms[J]. Advanced Photonics, 3, 024001(2021).

[5] He L D, Liu K X, He Z H et al. Three-dimensional holographic communication system for the metaverse[J]. Optics Communications, 526, 128894(2023).

[6] Gabor D. A new microscopic principle[J]. Nature, 161, 777-778(1948).

[7] Sohn I B, Choi H K, Yoo D et al. Three-dimensional hologram printing by single beam femtosecond laser direct writing[J]. Applied Surface Science, 427, 396-400(2018).

[8] Slinger C, Cameron C, Stanley M. Computer-generated holography as a generic display technology[J]. Computer, 38, 46-53(2005).

[9] Pi D P, Liu J, Wang Y T. Review of computer-generated hologram algorithms for color dynamic holographic three-dimensional display[J]. Light: Science & Applications, 11, 231(2022).

[10] Shimobaba T, Blinder D, Birnbaum T et al. Deep-learning computational holography: a review[J]. Frontiers in Photonics, 3, 854391(2022).

[11] Nishitsuji T, Shimobaba T, Kakue T et al. Review of fast calculation techniques for computer-generated holograms with the point-light-source-based model[J]. IEEE Transactions on Industrial Informatics, 13, 2447-2454(2017).

[12] Tsang P W M, Poon T C, Wu Y M. Review of fast methods for point-based computer-generated holography[J]. Photonics Research, 6, 837-846(2018).

[13] Zhao Y, Cao L C, Zhang H et al. Accurate calculation of computer-generated holograms using angular-spectrum layer-oriented method[J]. Optics Express, 23, 25440-25449(2015).

[14] Zhang H, Cao L C, Jin G F. Computer-generated hologram with occlusion effect using layer-based processing[J]. Applied Optics, 56, F138-F143(2017).

[15] Zhang Y P, Fan H X, Wang F et al. Polygon-based computer-generated holography: a review of fundamentals and recent progress[J]. Applied Optics, 61, B363-B374(2022).

[16] Matsushima K, Shimobaba T. Band-limited angular spectrum method for numerical simulation of free-space propagation in far and near fields[J]. Optics Express, 17, 19662-19673(2009).

[17] Li H, Sang X Z, Zhao L M et al. Optimized layered method for real-time interactive holographic display based on ray-tracing technique[J]. Optical Engineering, 59, 102408(2020).

[18] Bleha W P, Lei L A. Advances in liquid crystal on silicon (LCOS) spatial light modulator technology[J]. Proceedings of SPIE, 8736, 87360A(2013).

[19] Rudolf B, Du Y, Turtaev S et al. Thermal stability of wavefront shaping using a DMD as a spatial light modulator[J]. Optics Express, 29, 41808-41818(2021).

[20] Hsueh C K, Sawchuk A A. Computer-generated double-phase holograms[J]. Applied Optics, 17, 3874-3883(1978).

[21] Gerchberg R W. A practical algorithm for the determination of plane from image and diffraction pictures[J]. Optik, 35, 237-246(1972).

[22] Chakravarthula P, Peng Y F, Kollin J et al. Wirtinger holography for near-eye displays[J]. ACM Transactions on Graphics, 38, 213(2019).

[23] Gopakumar M, Kim J, Choi S et al. Unfiltered holography: optimizing high diffraction orders without optical filtering for compact holographic displays[J]. Optics Letters, 46, 5822-5825(2021).

[24] Vaswani A, Shazeer N, Parmar N et al. Attention is all you need[EB/OL]. https:∥arxiv.org/abs/1706.03762

[25] Han K, Wang Y H, Chen H T et al. A survey on vision transformer[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45, 87-110(2023).

[26] Li Z W, Liu F, Yang W J et al. A survey of convolutional neural networks: analysis, applications, and prospects[J]. IEEE Transactions on Neural Networks and Learning Systems, 33, 6999-7019(2022).

[27] Du G T, Cao X, Liang J M et al. Medical image segmentation based on U-Net: a review[J]. Journal of Imaging Science and Technology, 64, 020508(2020).

[28] Xu W N, Fu Y L, Zhu D M. ResNet and its application to medical image processing: research progress and challenges[J]. Computer Methods and Programs in Biomedicine, 240, 107660(2023).

[29] Shi L, Li B C, Kim C et al. Towards real-time photorealistic 3D holography with deep neural networks[J]. Nature, 591, 234-239(2021).

[30] Wu J C, Liu K X, Sui X M et al. High-speed computer-generated holography using an autoencoder-based deep neural network[J]. Optics Letters, 46, 2908-2911(2021).

[31] Agustsson E, Timofte R. NTIRE 2017 challenge on single image super-resolution: dataset and study[C], 1122-1131(2017).

[32] Peng Y F, Choi S, Padmanaban N et al. Neural holography with camera-in-the-loop training[J]. ACM Transactions on Graphics, 39, 185(2020).

[33] Shui X H, Zheng H D, Xia X X et al. Diffraction model-informed neural network for unsupervised layer-based computer-generated holography[J]. Optics Express, 30, 44814-44826(2022).

[34] Dong Z X, Xu C, Ling Y Y et al. Fourier-inspired neural module for real-time and high-fidelity computer-generated holography[J]. Optics Letters, 48, 759-762(2023).

[35] Liu Q W, Chen J, Yao Y W et al. Frequency aware high-quality computer-generated holography via multilevel wavelet learning and channel attention[J]. Optics Letters, 49, 5559-5562(2024).

[36] Yuan G, Zhou M, Liu F et al. Physics-aware cross-domain fusion aids learning-driven computer-generated holography[J]. Photonics Research, 12, 2747-2756(2024).

[37] Zhou Z W, Rahman Siddiquee M M, Tajbakhsh N et al. UNet++: a nested U-Net architecture for medical image segmentation[M]. Deep learning in medical image analysis and multimodal learning for clinical decision support, 11045, 3-11(2018).

[38] Wei Y R, Chen Y Y, Zhou M et al. Speckle-free holography with a diffraction-aware global perceptual model[J]. Photonics Research, 12, 2418-2423(2024).

[39] Yang T, Lu Z X. Holo-U2Net for high-fidelity 3D hologram generation[J]. Sensors, 24, 5505(2024).

[40] Yang L, Xu S Y, Yang C Z et al. High-quality computer-generated holography based on vision mamba[J]. Optics and Lasers in Engineering, 184, 108704(2025).

[41] Liu K X, Wu J C, He Z H et al. 4K-DMDNet: diffraction model-driven network for 4K computer-generated holography[J]. Opto-Electronic Advances, 6, 220135(2023).

[42] Jin Z W, Ren Q Y, Chen T et al. Vision transformer empowered physics-driven deep learning for omnidirectional three-dimensional holography[J]. Optics Express, 32, 14394-14404(2024).

[43] Dong Z X, Xu C, Tang Y Q et al. Vision transformer-based, high-fidelity, computer-generated holography[J]. Proceedings of SPIE, 12443, 1244307(2023).

[44] Zhong C L, Sang X Z, Yan B B et al. Real-time high-quality computer-generated hologram using complex-valued convolutional neural network[J]. IEEE Transactions on Visualization and Computer Graphics, 30, 3709-3718(2024).

[45] Zhang Y Z, Cheng D W, Wang Y S et al. Real-time multi-depth holographic display using complex-valued neural network[J]. Optics Express, 33, 7380-7395(2025).

[46] Xu Z Q, Leng J M, Dai P et al. DSCCNet for high-quality 4K computer-generated holograms[J]. Optics Express, 33, 13733-13747(2025).

[47] Qin H F, Han C, Shi X et al. Complex-valued generative adversarial network for real-time and high-quality computer-generated holography[J]. Optics Express, 32, 44437-44451(2024).

[48] Imtiaz S M, Amgalan T, Hossain F M F et al. Advanced deep learning model for direct phase-only hologram generation using complex-valued neural networks[J]. Neurocomputing, 631, 129672(2025).

[49] Yu G W, Wang J, Yang H et al. Asymmetrical neural network for real-time and high-quality computer-generated holography[J]. Optics Letters, 48, 5351-5354(2023).

[50] Yuan G, Zhou M, Peng Y F et al. Error-compensation network for ringing artifact reduction in holographic displays[J]. Optics Letters, 49, 3210-3213(2024).

[51] Liu Q W, Chen J, Qiu B S et al. DCPNet: a dual-channel parallel deep neural network for high quality computer-generated holography[J]. Optics Express, 31, 35908-35921(2023).

[52] Yan X P, Liu X L, Li J Q et al. Recoding double-phase holograms with the full convolutional neural network[J]. Optics & Laser Technology, 174, 110667(2024).

[53] Yan X P, Liu X L, Li J Q et al. Generating multi-depth 3D holograms using a fully convolutional neural network[J]. Advanced Science, 11, e2308886(2024).

[54] Zheng H D, Peng J C, Wang Z et al. Diffraction model-driven neural network trained using hybrid domain loss for real-time and high-quality computer-generated holography[J]. Optics Express, 31, 19931-19944(2023).

[55] Yu T, Zhang S J, Chen W et al. Phase dual-resolution networks for a computer-generated hologram[J]. Optics Express, 30, 2378-2389(2022).

[56] Fang Q W, Zheng H D, Xia X X et al. Generating high-quality phase-only holograms of binary images using global loss and stochastic homogenization training strategy[J]. Optics & Laser Technology, 181, 112059(2025).

[57] Chen C, Lee B, Li N N et al. Multi-depth hologram generation using stochastic gradient descent algorithm with complex loss function[J]. Optics Express, 29, 15089-15103(2021).

[58] Quan J L, Yan B B, Sang X Z et al. Multi-depth computer-generated hologram based on stochastic gradient descent algorithm with weighted complex loss function and masked diffraction[J]. Micromachines, 14, 605(2023).

[59] Shi L, Li B C, Matusik W. End-to-end learning of 3D phase-only holograms for holographic display[J]. Light: Science & Applications, 11, 247(2022).

[60] Wang X, Liu X L, Jing T et al. Phase-only hologram generated by a convolutional neural network trained using low-frequency mixed noise[J]. Optics Express, 30, 35189-35201(2022).

[61] Zhu R Z, Chen L Z, Zhang H. Computer holography using deep neural network with Fourier basis[J]. Optics Letters, 48, 2333-2336(2023).

[62] Liu K X, Wu J C, Cao L C. High-quality and high-speed computer-generated holography via deep-learning-assisted bidirectional error diffusion method[J]. Optics Express, 32, 37342-37354(2024).

[63] Yan X P, Li J Q, Zhang Y N et al. Generation of multiple-depth 3D computer-generated holograms from 2D-image-datasets trained CNN[J]. Advanced Science, 12, e2408610(2025).

[64] Kavaklı K, Itoh Y, Urey H et al. Realistic defocus blur for multiplane computer-generated holography[C], 418-426(2023).

[65] Yang D, Seo W, Yu H et al. Diffraction-engineered holography: beyond the depth representation limit of holographic displays[J]. Nature Communications, 13, 6012(2022).

[66] Chakravarthula P, Tseng E, Srivastava T et al. Learned hardware-in-the-loop phase retrieval for holographic near-eye displays[J]. ACM Transactions on Graphics, 39, 186(2020).

[67] Dong Z X, Ling Y Y, Li Y et al. Motion hologram: jointly optimized hologram generation and motion planning for photorealistic 3D displays via reinforcement learning[J]. Science Advances, 11, eads9876(2025).

[68] Choi S, Gopakumar M, Peng Y F et al. Neural 3D holography: learning accurate wave propagation models for 3D holographic virtual and augmented reality displays[J]. ACM Transactions on Graphics, 40, 240(2021).

[69] Choi S, Gopakumar M, Peng Y F et al. Time-multiplexed neural holography: a flexible framework for holographic near-eye displays with fast heavily-quantized spatial light modulators[C], 1-9(2022).

[70] Zhou W B, Qu F F, Meng X Y et al. 3D-HoloNet: fast, unfiltered, 3D hologram generation with camera-calibrated network learning[J]. Optics Letters, 50, 1188-1191(2025).

[71] Gopakumar M, Lee G Y, Choi S et al. Full-colour 3D holographic augmented-reality displays with metasurface waveguides[J]. Nature, 629, 791-797(2024).

[72] Kavaklı K, Shi L, Urey H et al. Multi-color holograms improve brightness in holographic displays[C], 1-11(2023).

[73] Chen C, Nam S W, Kim D et al. Ultrahigh-fidelity full-color holographic display via color-aware optimization[J]. PhotoniX, 5, 20(2024).

[74] Xia X X, Yang F R, Wang W S et al. Investigating learning-empowered hologram generation for holographic displays with ill-tuned hardware[J]. Optics Letters, 48, 1478-1481(2023).

[75] Peng Y F, Choi S, Kim J et al. Speckle-free holography with partially coherent light sources and camera-in-the-loop calibration[J]. Science Advances, 7, eabg5040(2021).

[76] Mildenhall B, Srinivasan P P, Tancik M et al. NeRF: representing scenes as neural radiance fields for view synthesis[J]. Communications of the ACM, 65, 99-106(2021).

[77] Kang M, Wang F, Kumano K et al. Neural-radiance-fields-based holography[J]. Applied Optics, 63, G24-G29(2024).

[78] Wang R, Ai L Y, Wang Y H et al. Generating real-scene hologram through light field imaging and deep learning[J]. Journal of Optics, 26, 075706(2024).

[79] Ishii Y, Wang F, Shiomi H et al. Multi-depth hologram generation from two-dimensional images by deep learning[J]. Optics and Lasers in Engineering, 170, 107758(2023).

[80] Liu N H, Huang Z Z, He Z H et al. DGE-CNN: 2D-to-3D holographic display based on a depth gradient extracting module and ZCNN network[J]. Optics Express, 31, 23867-23876(2023).

[81] Fang Q W, Zheng H D, Xia X X et al. Diffraction model-driven neural network with semi-supervised training strategy for real-world 3D holographic photography[J]. Optics Express, 32, 45406-45420(2024).

[82] Song X L, Dong J Q, Liu M H et al. Real-time intelligent 3D holographic photography for real-world scenarios[J]. Optics Express, 32, 24540-24552(2024).

[83] Chang C L, Wang D, Zhu D C et al. Deep-learning-based computer-generated hologram from a stereo image pair[J]. Optics Letters, 47, 1482-1485(2022).

[84] Zhang S J, Ma H W, Yang Y et al. End-to-end real-time holographic display based on real-time capture of real scenes[J]. Optics Letters, 48, 1850-1853(2023).

[85] Chang C L, Zhu D C, Li J M et al. Three-dimensional computer holography enabled from a single 2D image[J]. Optics Letters, 47, 2202-2205(2022).

[86] Chang C L, Dai B, Zhu D C et al. From picture to 3D hologram: end-to-end learning of real-time 3D photorealistic hologram generation from 2D image input[J]. Optics Letters, 48, 851-854(2023).

[87] Yu H, Kim Y, Yang D et al. Deep learning-based incoherent holographic camera enabling acquisition of real-world holograms for holographic streaming system[J]. Nature Communications, 14, 3534(2023).

[88] Wang D, Li Z S, Zheng Y et al. Liquid lens based holographic camera for real 3D scene hologram acquisition using end-to-end physical model-driven network[J]. Light: Science & Applications, 13, 62(2024).

[89] Li Z S, Liu C, Li X W et al. Real-time holographic camera for obtaining real 3D scene hologram[J]. Light: Science & Applications, 14, 74(2025).

[90] Wang Y J, Chakravarthula P, Sun Q et al. Joint neural phase retrieval and compression for energy- and computation-efficient holography on the edge[J]. ACM Transactions on Graphics, 41, 110(2022).

[91] Hu C C, Yang G L, Xie H Y. 3D information transmission of a computer-generated hologram using a quantum compensation hybrid neural network[J]. Optics Express, 32, 23736-23749(2024).

[92] Yao Y X, Yang G L. An optimized scheme of computer-generated hologram compression with convolutional neural networks and genetic algorithm[J]. Proceedings of SPIE, 13548, 135480C(2025).

[93] Endo Y, Oikawa M, Wilkinson T D et al. Quantized neural network for complex hologram generation[J]. Applied Optics, 64, A12-A18(2025).

[94] Ishii Y, Shimobaba T, Blinder D et al. Optimization of phase-only holograms calculated with scaled diffraction calculation through deep neural networks[J]. Applied Physics B, 128, 22(2022).

[95] Ryu W J, Lee J S, Won Y H. Continuous depth control of phase-only hologram with depth embedding block[J]. IEEE Photonics Journal, 14, 8821807(2022).

[96] Asano Y, Yamamoto K, Fushimi T et al. Conditional neural holography: a distance-adaptive CGH generator[J]. Optics Express, 33, 16671-16683(2025).

[97] Liu X L, Yan X P, Wang X. The U-Net-based phase-only CGH using the two-dimensional phase grating[J]. Optics Express, 30, 41624-41643(2022).

[98] Lee S, Nam S W, Lee J et al. HoloSR: deep learning-based super-resolution for real-time high-resolution computer-generated holograms[J]. Optics Express, 32, 11107-11122(2024).

[99] Shiomi H, Blinder D, Birnbaum T et al. Deep hologram converter from low-precision to middle-precision holograms[J]. Applied Optics, 62, 1723-1729(2023).

[100] No Y, Lee J, Yeom H et al. H2HSR: hologram-to-hologram super-resolution with deep neural network[J]. IEEE Access, 12, 90900-90914(2024).

Tools

Get Citation

Copy Citation Text

Chongli Zhong, Xinzhu Sang, Binbin Yan. Techniques on Deep Learning‐Based Computer‐Generated Hologram (Invited)[J]. Acta Optica Sinica, 2025, 45(17): 1720014

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites

Paper Information

Category: Optics in Computing

Received: Jun. 3, 2025

Accepted: Jun. 23, 2025

Published Online: Sep. 3, 2025

The Author Email: Xinzhu Sang (xzsang@bupt.edu.cn), Binbin Yan (yanbinbin@bupt.edu.cn)

DOI:10.3788/AOS251197

CSTR:32393.14.AOS251197

Topics

laser devices and laser physics

Lasers and Laser Optics

Laser physics

laser manufacturing

Instrumentation, Measurement and Metrology