After decades of research innovation in high-resolution microdisplay light engines, compact imaging optics, high-speed communication and computation, and heavy industrial investment in advanced manufacturing technologies, augmented reality (AR) and virtual reality (VR) have become key enablers for metaverse, spatial computing, and digital twins, that have found widespread applications in smart healthcare, education, and smart manufacturing, just to name a few¹⁻⁵. VR is an immersive headset; the microdisplay is located near the focal length of the imaging lens so that the viewer can see magnified virtual images. By adding cameras, such a video pass-through mixed reality (MR) enables the user to interact with the surrounding. Both Apple Vision Pro and Meta Quest 3 are MR displays. On the other end, AR is an optical see-through device; the user can see the displayed digital images and the real world simultaneously⁶⁻¹².

To enable comfortable longtime wearing and high ambient contrast ratio, ideally the AR glasses should have a stylish formfactor, lightweight, high brightness, and low power consumption. To fulfill these goals, several light engines have been developed, such as liquid-crystal-on-silicon (LCoS), laser beam scanner, MEMS (micro-electro-mechanical systems), micro-OLED (organic light-emitting diode), and micro-LED, etc. The pros and cons of each technology have been analyzed in detail⁶⁻⁹. Jade Bird Display (JBD) demonstrated a very impressive full-color AR glasses using three red, green, and blue (RGB) micro-LED panels combined with an x-cube prism¹³. The pixel pitch is ~4 μm and the resolution elements are 640×480. The diagonal size of each panel is 0.33 cm (0.13 inch), and the projector volume is only 0.4 cc (cm³). The approach sets a new milestone in terms of compact light engine, but there are two major challenges remain to be overcome: 1) it requires accurate pixel registration for the three micro-LED panels, and 2) the radiation pattern of the RGB LEDs should be the same, otherwise, the color mixing will not be uniform at the projection screen. Moreover, as the resolution increases, the volume and power consumption will also increase proportionally, provided that the pixel pitch remains the same. For example, if the resolution increases from 640×480 to 1024×1024, then the panel size and power consumption will each increase by 3.4×.

On the other hand, field-sequential-color LCoS using RGB LEDs as the illumination light source has been used in Microsoft HoloLens¹⁴, Lumus Maximus¹⁵, and Magic Leap 2 AR headsets¹⁶, etc. The basic operation principles of LCoS have been elucidated in several review papers¹⁷⁻²¹ and will not be repeated here. By removing the lossy spatial color filters, such an LCoS offers tripled resolution density and optical efficiency as compared to the color filters based LCoS¹⁹. However, conventional LCoS uses a polarizing beam splitter (PBS) cube as an optical combiner, e.g., Google Glass, as Fig. 1(a) depicts, thus, the entire illumination system is relatively bulky (volume ~4 cc for the 1024×1024 panel). Recently, to reduce the volume to ~0.5 cc, Himax Display developed a Front-lit LCoS optics²² and Avegant proposed a waveguide approach²³. However, the tradeoff of the Himax approach is the reduced optical efficiency (~10% for a linearly polarized light) while the latter encountered a lower contrast ratio (CR~100:1) due to stray light. There is an urgent need to develop a compact LCoS system while keeping a high optical efficiency and high contrast ratio.

In this paper, to dramatically reduce the LCoS formfactor while maintaining a high optical efficiency, we propose a novel illumination system, including an in-coupling prism and a light guide plate (LGP) with multiple parallelepiped extraction prisms. Our simulation results demonstrate an excellent illuminance uniformity and nearly 36%–41% optical efficiency for a linearly polarized incident light. The estimated volume is about 0.25 cc for an 1024×1024 field-sequential-color LCoS panel (~4.4-μm pixel pitch), excluding the projection optics. Offering an ultracompact formfactor and a high optical efficiency, our novel LCoS light engine has potential to revolutionize the next-generation lightweight AR glasses.

To reduce the bulky formfactor while maintaining a high optical efficiency, we combine a thin LGP (T = 0.2, 0.4, 0.8 mm) with parallelepiped extraction prisms as the illumination optics for the LCoS light engine. The light emitted from the LED array is coupled into the LGP using an in-coupling prism shown in Fig. 1(b). Next, the in-coupled light is trapped inside the LGP due to total internal reflection (TIR) at the top and the bottom surfaces. Some of the trapped light enters the extraction prisms as shown in the enlarged figure while propagating along Z direction and the remaining light continues to propagate forward in the LGP. The light inside the extraction prism is reflected toward the bottom LCoS panel through another TIR at the tilted surface of the prism. Similar structure has been proposed to generate uniform illumination for liquid crystal display²⁴. The LCoS panel manipulates the polarization states pixel-by-pixel and reflects the incoming light back to LGP. Most of the reflected light with encoded information transmits through the LGP and the clean-up polarizer on the top, and finally enters the projection lens system (not shown here), which in turn will be coupled into the subsequent optical combiner of the AR system (not shown here). It should be mentioned here that the quarter-wave plate (QWP) in Fig. 1(b) is optional, depending on the employed LC mode. For example, if a normally black vertical alignment (VA) LCoS is used, then the circularly polarized light after the QWP helps circumvent the fringe field effects²⁵. On the other hand, the normally white MTN (Mixed-mode Twisted Nematic) LCoS can take either linearly or circularly polarized light²⁶. In Magic Leap 2, circular polarization is chosen to mitigate the stray light from surface reflection in the projection system¹⁶. Another advantage of MTN is its fast response time (~1 ms) and weak fringe field effect.

In our study, the emitted light from the LED array is assumed to possess Gaussian angular distribution with a full width at half maximum (FWHM) of ±16°. The desired angular intensity profile can be manipulated by additional beam shapers if needed. The luminous flux of the RGB LED array in our simulations is assumed to be 1 lumen. Thus, only 0.5 lumen is left after passing through a linear polarizer. To enhance light efficiency, polarization recycling using a reflective polarizer can be implemented at this stage. The refractive index of the LGP and the extraction prisms is assumed to be n = 2 in our first design because a higher refractive index helps to narrow the emission cone inside the material. These high-index glass materials have been commercialized by Corning and AGC. Later, our analyses will also extend to a lower refractive index n = 1.7 material because it is easier to be achieved by plastic. In Fig. 1(b), a right-angle prism is employed to couple the light into the LGP whose thickness is T = 0.4 mm. The index of this prism is also chosen to be n = 2 to ensure a good match with the LGP. The distance L between the in-coupling prism and the first extraction prism should be long enough to ensure a thorough light mixing. The width of the extraction prism is assumed to be W = 10 μm, which is significantly longer than the visible light wavelength to reduce diffraction effects. H represents the height of the extraction prism, P denotes the period of these prisms, and D is the distance between the LGP and the LCoS panel. The optimization process of these parameters will be further investigated in the following section. In addition to illumination optics, LCoS panel size is also essential to our simulations. Here, we assume the pixel pitch is 4.4 μm and resolution is 1024×1024, as a result, the active area of LCoS panel is 4.5 mm by 4.5 mm. Most normally white LCoS panels employ the 90° MTN cell because of its high reflectance (~80% after considering the fringe field effect)²⁷ and fast response time to mitigate color breakup. The reflectivity of the bottom pixelated aluminum mirror is about 90%, thus, the total LCoS reflectance is about 70% for simulating the bright-state performance at the voltage-off state (V = 0).

To ensure an excellent image performance for AR displays, the employed light engine must deliver uniform illuminance. Therefore, uniformity is one of the most important metrics during our simulations, and it is closely related to the spatial distribution of the extraction prisms. We first consider the configuration where all the extraction prisms are periodically distributed along Z direction at the bottom of LGP. Here, we define the aperture ratio as W/P, where W is the width of the prism and P is the period. The aperture ratio determines the amount of light can be extracted from the LGP. The first simulation was conducted for the periodically distributed extraction prisms with W/P = 0.5. An illuminance detector was positioned near the top of the polarizer and the results are depicted in Fig. 2.

The received illuminance gradually decreases when it comes to the distal side along Z direction. The luminous flux of light trapped in the LGP keeps decreasing while propagating, because some light is already extracted out by the previous extraction prisms, resulting in a weaker illuminance at the distal side if the aperture ratio remains the same in the whole area. Therefore, the pitch of the extraction prisms must be optimized in order to achieve uniform illuminance at the top.

Before diving into optimizing the extraction prisms, we investigated the light propagation inside the LGP first. Here, for simplicity, light travelling in the perpendicular direction with respect to the entrance surface of the in-coupling prism is shown in Fig. 3(a). The incident light hits the top surface and is then reflected to the bottom surface of the LGP due to TIR. The length of the interaction area at the top and bottom surfaces along Z direction is defined as TIR length. The illuminance inside each TIR area is supposed to be uniform. Thus, the extraction prisms can be divided into several zones and the prisms in each zone share the same aperture ratio as Fig. 3(b) shows. We simulated the transmission and extraction efficiency with different aperture ratios to better understand the optical performance. Results are plotted in Fig. 3(c). In these tests, no light source was used at the in-coupling prism. The transmission was measured by disposing a light source under the extraction prism and the transmitted light was measured at the top of LGP. The efficiency is usually lower than 100% as some of the light entering the prisms is trapped inside the LGP, which is a reversed process as compared to the extraction process discussed earlier. The transmission reaches 100% when W/P = 1, which means no prisms. Next, we measured the extraction efficiency of the extraction prisms. In this measurement, we placed a light source at the in-coupling prism and placed an illuminance detector at the bottom of the extraction prisms. As expected, the efficiency is linearly dependent on the aperture ratio. When W/P = 1, it means no extraction prism so that the light is trapped in the LGP due to TIR. The efficiency at different aperture ratios is simply defined as the product of transmission and extraction. According to the obtained efficiency, a larger aperture ratio (except 1) is preferred as it provides a higher efficiency. However, such a large W/P imposes fabrication difficulties as the valley between two neighboring prisms becomes too narrow. Therefore, we choose an aperture ratio ≤0.5 for further optimizations.

The necessity of multiple zones has been discussed above and the length of the divided zones will be investigated in the following. It is obvious that the zone length is closely related to the TIR length introduced in Fig. 3(a). TIR length is solely determined by the LGP thickness because we employed a right-angle prism as the in-coupling prism. The TIR length is 0.8 mm if we choose the LGP thickness T = 0.4 mm. The LGP thickness effect will be discussed later. Here, we normalize different zone lengths using the TIR length without losing generality. The detector was located at the bottom of the extraction prism, and we only paid attention to the extraction efficiency in this part. The simulated cross-sectional results of LCoS systems with only one extraction zone (W/P = 0.5) are plotted in Fig. 3(d), and lines with different colors represent the simulation results with different normalized zone lengths. We find that the slice diagram remains good uniformity when the zone length is shorter than the TIR length. The uniformity degrades dramatically as the zone length increases, especially when the normalized zone length exceeds 120%. Next, we studied the LCoS systems with two extraction zones (W/P = 0.33 and 0.5) to examine how the previous zone affect the subsequent zone. In Fig. 3(e), we find both shorter and longer zone lengths lead to an undesirable nonuniformity. Based on our simulation results, the ideal zone length should be approximately equal to the TIR length, which is 0.8 mm in the present configuration.

Therefore, we divide the extraction prisms into 5 zones with a zone length of 0.9 mm for each zone, because the active LCoS panel length is 4.5 mm in Z direction. Next, the aperture ratios of each zone shall be determined to enable uniform illuminance. The input, extraction, and output of each zone can be estimated based on the simulation results in Fig. 3(c) if we assume the aperture ratios of the 5 zones to be A1, A2, A3, A4, and A5 as summarized in Table 1.

Here, E(A) and Effi(A) represents the extraction function and the efficiency function of the aperture ratio, respectively. The output of different zones is expected to be identical to ensure uniform illuminance. Multiple sets of solutions exist, and we set A5 = 0.5 manually to obtain one set of solutions as shown in Fig. 3(f). The green arrows in Fig. 3(f) indicate the process to find the corresponding aperture ratios of the other four zones. The calculated aperture ratios are [0.1419, 0.1737, 0.2144, 0.3151, 0.5] for the five zones.

Simulations with the calculated aperture ratios were conducted and the results are illustrated in Fig. 4. The illuminance distribution was recorded by placing a detector near the top of the LCoS system. Figure 4(a) shows the results using the aperture ratios calculated above. However, the distal side shows a higher illuminance, and the cross-sectional diagram is plotted as the light blue line in Fig. 4(d). The nonuniformity originates from the reflection at the end surface of the LGP, as the remaining light was bounced back after hitting the end surface due to TIR. Such a mechanism is verified by adding an absorbing coating on the end surface, and the illuminance with a better uniformity is illustrated in Fig. 4(b). The simulation results indicate the effectiveness of our calculations on the aperture ratios. In addition to absorbing coating, fine-tuning the aperture ratios also helps to achieve a better uniformity. The optimized aperture ratios for the five zones are [0.155, 0.1737, 0.2144, 0.25, 0.35] and the simulated results are shown in Fig. 4(c). Under such conditions, the simulated optical efficiency is about 36.38% for a polarized input light. The cross-sectional diagrams of the three device configurations are plotted in Fig. 4(d).

Recall in Fig. 1(b), we define a variable L as the distance from the in-coupling prism to the first extraction prism, and D as the distance between the LGP and LCoS panel. The effects of these two parameters will be investigated here. L is related to the light mixing process and is normalized to the TIR length in our simulations. The cross-sectional diagram of the simulated results with different L values are depicted in Fig. 5(a). The illuminance of different L values is shifted intentionally for easier observation. We find that L should be long enough (comparable with TIR length) to ensure a thorough light mixing before entering the extraction prisms. Besides, simulations with different D values are also conducted and the results are shown in Fig. 5(b) and 5(c). The cross-section of the illuminance maintains an excellent uniformity even if D increases from 0.05 mm to 0.55 mm. However, the luminous intensity shows a strong dependence on the D value. When the LCoS is close to the LGP, a valley emerges near 0° region, corresponding to the light in the vertical direction. The origin of this valley is explained in Fig. 5(d). The reflected light along vertical direction from the LCoS enters the extraction prism for a second time and is trapped in the LGP (red arrows). The remaining light travelling in off-axis direction passes through the LGP without being reflected by the extraction prisms (blue arrows). Therefore, a lager D value aids to narrowing the width of the valley because the light spreads in the horizontal direction while travelling. Fortunately, the LCoS panel has a cover glass on the top, whose thickness (0.3–0.4 mm) is larger than the desired D to provide a good luminous intensity. However, a too large D between the LCoS panel and LGP would compromise the formfactor and diminish the illuminance at edges, as illustrated in Fig. 5(b).

In addition to uniform illuminance, high contrast ratio of the LCoS display is also essential to ensure comfortable viewing experience for AR glasses in the dark ambient. The height of extraction prisms (H) is closely related to the stray light generations as illustrated in Fig. 6(a) and 6(b). A larger or a smaller H value results in TIR at the bottom surface of the extraction prisms, which in turn generates stray light. To investigate its influence on SNR, we define the height-to-width ratio of the extraction prisms as H/W, where W is the width of prism. In our simulations, we take W = 10 μm to avoid severe diffractions. A checkerboard pattern shown in Fig. 6(c) is displayed on the LCoS panel to simulate the ANSI contrast ratio of the LCoS system. The imaging result is obtained using an ideal lens with a focal length of 10 mm. Both object and image distances are about 20 mm to replicate the checkerboard pattern on the image plane, and the resulted image is illustrated in Fig. 6(d). Then the ANSI contrast is calculated with the simulated illuminance distribution on the image plane and the results are shown in Fig. 6(e), which indicate a significant dependence of ANSI contrast on H/W. A peak ANSI contrast of 875 is achievable when H/W = 2.4. In addition to the reflection inside the extraction prisms, TIR at end surface also generates stray lights as the reflected light travels backward along the negative Z direction, as shown in Fig. 6(f), which is opposite to the original design. To study the stray light influences on ANSI contrast, we added an absorbing coating on the end surface and the simulated ANSI contrast is 1551, which is almost doubled compared to the results without absorbing coating. However, absorbing the light incident onto the end surface affects the illuminance uniformity as shown in Fig. 6(g). Fortunately, it can be optimized by fine-tuning the aperture ratios of the extraction prisms, as Fig. 4(c) depicts.

Simulations on the illumination systems with different LGP thickness are also conducted after optimizing the SNR of the illumination optics. The extraction prisms are divided into 5 zones for the LGP thickness T = 0.4 mm as mentioned above. Similarly, 3 zones are needed if T = 0.8 mm and 11 zones are required if T = 0.2 mm. The optical efficiency for the 0.8-mm and 0.2-mm LGP is 29.58% and 41.22%, respectively. Hence, a smaller thickness helps enhance the optical efficiency of the illumination system because a thinner LGP facilitates more occurrences of TIR, ensuring thorough interactions between the trapped light and the extraction prisms. However, the aperture ratios in some of the 11 zones might be too small and the distribution of the extraction prisms can be too sparse, requiring a larger space (i.e., larger D value) between the LGP and LCoS panel, which compromises the formfactor. Moreover, a thinner LGP reduces the size of the light source. Specifically, the height of the front-lit LED source is limited to 0.28 mm if T = 0.2 mm while it allows a larger size of 0.56 mm when T = 0.4 mm. A larger LED chip size is preferred due to its higher efficiency. The efficacy of the LED chips decreases as the LED size gets smaller. Therefore, the subsequent simulations will continue to be based on the 0.4-mm thick LGP.

To compare the volume with currently available LCoS systems, the width, length, and thickness of the LCoS panel alone including the peripheral package area is about 9 mm, 13.7 mm, and 1.5 mm, as reported by Himax²¹. After adding the front-lit illumination optics, the height increases from 1.5 mm to 3.8 mm, so the overall volume of the LCoS module is about 0.47 cc (0.9×1.37×0.38). In our design, the total thickness after adding the proposed illumination optics is about 2 mm (including LCoS panel, extraction prisms, LGP, and polarizer). Therefore, the LCoS module volume of our design, excluding the projection optics, is around 0.25 cc (0.9×1.37×0.2), which is ~2× smaller, yet the efficiency is ~4× higher, than Himax’s results. This indicates that our TIR based LGP with extraction prisms is more compact and more efficient than Himax’s light mixing approach.

In addition to illuminance and SNR, special attention must be paid to the potential image distortion caused by the extraction prisms. For this purpose, a flat panel display without any illumination optics was imaged using a perfect lens with a focal length of 10 mm. Both object and image distance are 20 mm to replicate the object image shown in Fig. 7(a) on the image plane, and the resulted image of the flat panel display is illustrated in Fig. 7(b). Similar imaging process was conducted for our new compact LCoS system, and the imaging results are depicted in Fig. 7(c). No image distortion is observed by comparing the images produced by these two light engines.

To study the color performance of our new compact LCoS system, we conduct simulations with RGB wavelengths. Figure 8(a) shows the refractive index dispersion of Corning ARS 2.0 glass, and Fig. 8(b) and 8(c) shows the cross-sectional diagrams of illuminance and luminous intensity, respectively. More detailed simulation results regarding the illuminance and intensity of different colors are illustrated in Fig. 8(d−i). The excellent color performance is attributed to the novel design, the width of the extraction prisms is significantly larger than the wavelength. It is worth noting that although only three single-wavelength RGB colors (R = 630 nm, G = 550 nm, B =450 nm) are used in our simulations, the achromatic behavior indicates that our proposed LCoS system works equally well for a broadband light source, such as RGB LEDs or white LEDs.

As discussed earlier, we have explored the device design with a high-index (n = 2) glass. However, the micrometer-scale fabrication process of the light extraction prisms on such a high-index glass material may not be cost effective. For plastic materials, their highest achievable refractive index is in the 1.7–1.8 range, which has been demonstrated in AR displays. Therefore, it is important to investigate the system performance with a lower index (n = 1.7) material. Simulations were first conducted by simply changing the refractive index of the in-coupling prism, LGP, and extraction prisms from 2.0 to 1.7 without any further optimization as shown in Fig. 9(a). The simulated cross-sectional diagrams of illuminance and luminous intensity are represented by the red lines in Fig. 9(c) and 9(d). Illuminance remains acceptable while the intensity exhibits an abrupt decrease in the vicinity of 20°. To understand the underlying origins, we calculated the TIR angles from materials with different refractive index into air, and the results are illustrated by the black line in Fig. 9(e). All the incident angles of the emitted light from the light source should be larger than the TIR angle to ensure all the light can be trapped inside the LGP in our design. The luminous intensity drops to nearly 0 at 30° when the emission cone exhibits a Gaussian angular distribution with FWHM = ±16°, as used in our simulations. The corresponding half cone inside the material was calculated and illustrated by the red line in Fig. 9(e). Blue line is the summation of the other two solid lines and represents the required minimum angle θ to ensure the occurrences of TIR for all the emitted light in different materials. The required minimum θ for the n = 1.7 material is much larger than 45° as shown in Fig. 9(e), which is reason that the luminous intensity is cut off at ~20°. The intensity issue can be addressed by using an optimized in-coupling prism with a new θ = 58° as shown in Fig. 9(b). Simulation results of the cross-section diagrams after using a new prism are depicted by the dark blue lines in Fig. 9(c) and 9(d). The sharp decrease disappears but sidebands emerge, which degrades the performance of AR displays. Moreover, the system optical efficiency is around 26.90%, which is lower than the previous system (Fig. 9(a)) with an optical efficiency of 36.38% using the n=2 material. Therefore, a higher refractive index is preferred for both higher efficiency and better luminous intensity.

We demonstrate an ultracompact LCoS system with a volume of merely 0.25 cc excluding the projection lens, while keeping a reasonably high optical efficiency (36%–41% for a linearly polarized RGB LED light). The proof-of-concept design is conducted through simulations. The proposed ultracompact illumination system consists of an in-coupling prism, a LGP, and multiple parallelepiped extraction prisms. The emitted light from LED array is coupled into the LGP by the in-coupling prism and trapped inside the LGP due to TIR. The light enters the extraction prisms and is reflected toward to the LCoS panel during its propagation along the LGP. Then, the LCoS panel underneath manipulates the phase retardation and reflects the light back to the extraction prisms and LGP. The light encoded with pixel information passes through the illumination system and is collected by the projection optics for further AR applications.

Optimizations on the system configuration and size of each component have been conducted to achieve an excellent illuminance uniformity and high SNR, which is equal or better than the contrast ratio of the LCoS panel. Besides, the outstanding color performance is demonstrated by taking the refractive index dispersion of the glass material into consideration. Additionally, plastic material with a lower refractive index n = 1.7 can also be employed for lowering the cost. Its optical performance is acceptable, although a higher-index material is preferred.

The proposed system exhibits an impressive optical efficiency of 36.38% for a polarized light using a 0.4-mm thickness LGP with a refractive index n = 2. In comparison, other existing LCoS illumination designs can provide an optical efficiency of about 10% for a linearly polarized light. In our design, a higher optical efficiency of 41.22% can be achieved with a thinner (0.2 mm) LGP. The depth of the illumination system for a 4.5-mm by 4.5-mm LCoS panel (resolution 1024×1024) is reduced from 4.5 mm (using a PBS cube) to 0.4 mm or even 0.2 mm in our design. Such a slim formfactor and high optical efficiency are expected to make a big impact to next-generation lightweight and low power AR glasses.