Acta Optica Sinica, Volume. 45, Issue 17, 1720016(2025)

Portrait Content Generation Technologies for 3D Light Field Displays (Invited)

Sheng Shen, Xinzhu Sang*, Shujun Xin, and Binbin Yan
Author Affiliations
  • School of Electronic Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China
  • show less

    Significance

    As creatures in the three-dimensional world, human beings mainly observe it through their eyes. When the human eye observes objects, there is parallax, which the brain can synthesize to perceive information with spatial depth. In computer graphics, graphic rendering can convert objects in 3D space into images on a 2D screen through model views and perspective transformations. Due to the effect of objects appearing larger when closer and smaller when farther away produced by perspective transformation, as well as the occlusion and shadows between objects, combined with the prior knowledge of the human brain, people can still sense the depth of objects when watching 2D screens. However, from the perspective of information, compressing the information of the 3D world onto a 2D screen for display still leads to information loss. This will also lead to perceptual deviations in the human eye during observation, and at the same time, it is difficult to break through the limitations of 2D display to directly obtain depth information. In principle, unlike 2D displays, 3D light field displays can provide viewers with information from multiple viewpoints and the correct front-back relationships between objects.

    With the advancement of display technology, 3D display technology has also made significant progress. At present, commercially available 3D display technologies can mainly be classified into assistive 3D displays that require wearing external devices and naked-eye 3D displays that do not require external devices. Among them, the greatest advantage of naked-eye 3D display is that viewers can enjoy the 3D display effect without relying on any peripheral devices. At present, naked-eye 3D display technology has made significant breakthroughs both in theory and practice. This direction has not only received extensive attention in the academic circles but also brought many applicable products to the industrial sector, covering various fields such as education, healthcare, national defense, and advertising.

    In recent years, portrait content generation technology has been a key research focus in the field of graphics and image science. The advantage of high-quality portrait content generation technology lies in its provision of more realistic, personalized, and creative digital experiences for various application fields, promoting the development of virtualization and digital technologies in all aspects of society. Thanks to the rapid development of portrait content generation technology and light field display technology, the application fields that combine portrait content generation technology with 3D light field display technology are very extensive. For instance, in the field of communication, the advantage of light field communication lies in its ability to utilize the characteristics of light field displays to provide callers with a more realistic and immersive communication experience. In the field of education, 3D light field display can present object models with correct occlusion relationships, helping learners obtain correct spatial information when observing. In the field of cultural media, 3D light field display can be applied to the production and display of cultural relics or advertisements, providing viewers with a more vivid three-dimensional effect. Therefore, 3D light field portrait display has an urgent application demand and broad market prospects in fields such as communication, education, and industry. Combining 3D light field display with portrait content generation technology will become the main development direction in the future. Enhancing the wide application of 3D portraits in 3D light field display is mainly achieved through two aspects: the realism of portrait content and computational efficiency. Therefore, this paper reviews the content generation technology of portrait scenes used in 3D light field display.

    Progress

    Three-dimensional light field display technology, through the reconstruction of multi-angle light field information, provides a highly immersive visual experience for the remote presentation of three-dimensional human figures. Its effect is highly dependent on the generation of high-quality three-dimensional human figure content. This paper studies the 3D portrait content generation technology for 3D light field display, summarizes the technological progress based on traditional modeling and deep learning (Fig. 1), and points out the significant advantages of deep learning-based methods in the generation of dynamic portrait continuous viewing angles. The advantages and disadvantages of typical 3D portrait content generation methods in recent years and the quality of the generation results are summarized and compared (Table 1). The specific applications of 3D portrait generation technology in light field display are analyzed, mainly including the optimization of light field portrait re-illumination and the driving of rapid reconstruction. Among them, portrait re-illumination mainly focuses on the portrait re-illumination technology based on parameter combination (Fig. 6) and the portrait re-illumination technology based on network reasoning (Fig. 8). The rapid reconstruction driver of human figures mainly focuses on the generation technology of human figure content based on voice drive (Fig. 10) and parametric drive (Fig. 12). The method based on deep learning can not only reduce the cost of traditional modeling but also ensure a lower reconstruction error. Moreover, the re-illumination of human figures and the real-time reconstruction driving technology significantly enhance the realism and interactivity of the three-dimensional light field content. Future research needs to continue to deepen exploration in aspects such as high resolution and real-time generation of a large number of viewpoints.

    Conclusions and Prospects

    Centering on the requirements of 3D light field display, this paper introduces two technologies for generating 3D portrait content for three-dimensional light field display. Static portrait generation relies on multi-view fusion and neural implicit representation, while dynamic portrait generation is expressed through the combination of deformation networks and explicit Gaussian distribution. Both have made significant progress in terms of accuracy and efficiency. In the application of three-dimensional light field display, the reillumination technology combined with ambient light estimation optimizes scene adaptability and enhances the realism of the generated content. The rapid reconstruction-driven technology has improved the efficiency of content generation and provided feasibility for real-time interaction. However, the existing technology still faces challenges such as insufficient dynamic detail modeling and the inability to generate a large number of viewpoints in real time. Future research needs to explore lightweight generation algorithms, intelligent frame interpolation and super-resolution algorithms, to promote the wide application of 3D portrait content generation technology in virtual reality, remote 3D light field video communication and other fields.

    Keywords
    Tools

    Get Citation

    Copy Citation Text

    Sheng Shen, Xinzhu Sang, Shujun Xin, Binbin Yan. Portrait Content Generation Technologies for 3D Light Field Displays (Invited)[J]. Acta Optica Sinica, 2025, 45(17): 1720016

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Optics in Computing

    Received: Jun. 4, 2025

    Accepted: Aug. 14, 2025

    Published Online: Sep. 3, 2025

    The Author Email: Xinzhu Sang (xzsang@bupt.edu.cn)

    DOI:10.3788/AOS251216

    CSTR:32393.14.AOS251216

    Topics