The human visual system (HVS) exploits a wide range of cues for the perception of three-dimensional (3D) space and shapes. Many different 3D display technologies have been developed over the years, most of which render a subset of these visual cues to facilitate 3D perception. For instance, the well-established stereoscopic 3D displays (S3Ds) render a pair of two-dimensional (2D) perspective images, one for each eye, with binocular disparities and other pictorial depth cues of a 3D scene seen from two slightly different viewing positions. Although they can create compelling depth perceptions, the S3Ds are unable to render correct focus cues for 3D scenes, including accommodation and retinal blur effects and thus are subject to a well-known vergence-accommodation conflict (VAC) problem. The VAC may be manifested as the mismatch between the depths of eye accommodation and vergence or the mismatch between the retinal blurring effects of rendered and real-world 3D scenes. These conflicts are considered the key contributing factors to various visual artifacts associated with viewing S3Ds, such as distorted depth perception and visual discomfort.

Several display methods that are potentially capable of resolving the VAC problem have been explored, among which a light field display aims to render the perception of a 3D scene by reproducing the geometric light rays apparently emitted by the 3D scene in different directions and thus deliver a visual experience that is natural and comfortable for the HVS. The geometrical light rays may be reconstructed through a stack of discrete focal planes placed at different depths or through an optics array angularly sampling the different directions of the light rays viewed from different eye positions. The light field display method is considered one of the most promising 3D display technologies capable of addressing the VAC problem while offering great flexibility and scalability.

Over the last decades, many different methods have been explored and demonstrated to render light field effects and achieved different degrees of fidelity. Light field displays may be implemented in the form of direct-view displays or head-mounted displays. With exploding interests in virtual and augmented reality applications, various light field display methods have been explored for head-mounted displays, known as light field head-mounted displays (LF-HMDs). In LF-HMDs, the display source and the optics are worn on the viewer's head. The light field of a 3D scene may be sampled by a stack of discrete focal planes placed at different depths along the visual axis, known as multi-focal-plane HMDs (MFP-HMDs), or by an array of optical elements sampling the angular directions of the light rays apparently emitted by the 3D scene, known as integral imaging (InI)-based HMDs (InI-HMDs), or a stack of multi-layer spatial light modulators which computationally reconstruct the four-dimensional light field functions, known as computational multi-layer HMDs.

In multi-focal-plane LF-HMDs, the light field of a 3D scene is sampled by a stack of discrete focal planes placed at different depths along the visual axis, as illustrated in Fig. 2. When the focal planes are adequately dense so that each spatial location of a 3D scene is uniquely sampled by a pixel on a corresponding plane, each pixel is known as a voxel, and the configuration is known as a volumetric display, which can be viewed from a large range of viewing points. In addition, when the focal planes are sparse, and the adjacent focal planes are largely separated, an extended 3D volume is divided into multiple focal zones by these focal planes, each of which renders the 2D projection of a sub-volume of the 3D scene centered at the corresponding depth of the focal plane from a fixed viewpoint. These 2D projections additively reconstruct the light field of the scene seen from the given viewing point. In this configuration, the light rays for reconstructing a 3D scene are sampled from the same viewpoint with ray positions varying in depth. Such a configuration is known as a fixed viewing point volumetric display. Such a simplified sampling mechanism gains the benefit of employing a small number of focal planes adequate for rendering a large depth volume with high spatial resolution at the cost of relatively low depth resolution.

In an InI-based LF-HMDs, the light field of a 3D scene is reconstructed by angularly sampling the geometrical light rays apparently emitted by a 3D scene through 2D optical array elements such as a lenslet array or pinhole array. As illustrated in Fig. 3, a simple InI-based LF-HMD consists of a micro-display panel and a micro-lens array (MLA) placed directly in front of a viewer's eye. The display renders a set of 2D elemental images, each of which represents a different perspective of a 3D scene. The light fields of the reconstructed 3D scene are directly coupled into a viewer's eye, and thus the view window through which the eye of a viewer receives the light fields is confined to a small region.

A computational multi-layer LF-HMD is based on the same principles as compressive displays, which samples the directional rays through multi-layers of pixel arrays illuminated by either a uniform or directional backlight. As illustrated in Fig. 4, the light field of a 3D scene is computationally decomposed into several attenuation masks representing the transmittance or reflectance of each layer of the light attenuators. The intensity value of each light ray entering the eye from the backlight is the product of the pixel values of the attenuation layers at which the ray intersects. Due to the collimated nature of the reconstructed light rays, a computational multi-layer display renders the light fields of a 3D scene through the integral sum of the directional light rays.

The author provides a comprehensive overview of different approaches and recent development of these three different optical architectures for LF-HMDs.

Though significant progress has been made over the last decade in developing LF-HMD technology, some significant challenges need to be overcome to fully realize light field displays. Besides the obvious challenge of achieving a glass-like form factor, some of the big challenges are huge computational requirements for rendering the light field, large data bandwidths needed for transmission to the display, significant power consumption for the system, and high image quality in terms of brightness, contrast, color gamut, dynamic range, and spatial and temporal resolution expected by consumers because of the advanced state of 2D displays. Furthermore, with the rapid development of hardware, software, computing, and electronics, we remain very optimistic about coming through all of these technical hurdles.