Augmented reality (AR) helmet-mounted displays (HMDs) hold significant promise for applications in education, healthcare, industry, and military fields, emerging as a crucial platform for next-generation human-computer interaction. The design and optimization of optical systems are critical for enhancing AR-HMD performance. Traditional optical design methods primarily involve identifying initial structures from patent literature and subsequent optimization, which not only demand substantial human resources but also rely heavily on designers’ personal experience. This makes it challenging to meet the rapid development needs of AR technology. Given the successful application of deep learning technology across various fields, integrating it into optical system design has become increasingly feasible. In this paper, we propose a design method combining optical simulation software with deep learning, aiming to automate the construction of AR optical systems, thus improving design efficiency and reducing reliance on designer experience.

In this paper, we utilize xy polynomial freeform surfaces based on quadric surfaces to design an optical structure that combines freeform prisms and compensation prisms. This structure integrates the display light path (one total internal reflection, one reflection, and two transmissions) with the real-world light path. For dataset acquisition, initial structures are created via Python-Zemax software interaction, with half-XFOV ($F_{\mathrm{h}\mathrm{a}\mathrm{l}\mathrm{f}\mathrm{?}X}$), half-YFOV ($F_{\mathrm{h}\mathrm{a}\mathrm{l}\mathrm{f}\mathrm{?}Y}$), and F-number (F) as key input parameters, with their respective ranges determined. Several base systems with good imaging performance and reasonable optical structure are obtained by optimizing the prism structures’ edges and total internal reflection. Based on these base systems, system samples covering the entire input parameter range are generated through gradual parameter variation (step size 0.2). The system performance is evaluated using merit function results and the root-mean-square (RMS) radius of each field. Finally, samples with optical structure errors are filtered out, leaving only those that meet imaging quality requirements (Fig. 3). The deep neural network training employs a multi-layer perceptron (MLP) structure (Fig. 4). The input layer consists of three features ($F_{\mathrm{h}\mathrm{a}\mathrm{l}\mathrm{f}\mathrm{?}X}$, $F_{\mathrm{h}\mathrm{a}\mathrm{l}\mathrm{f}\mathrm{?}Y}$, F), while the output layer includes the optical system’s structural parameters (radius of curvature, thickness, conic value, decentration, tilt degree, and freeform surface coefficients). The network comprises four hidden layers, uses a tanh activation function, and employs the MSELoss function and Adam optimizer for training. System verification is carried out by using the trained model to predict optical system structures, importing the predicted results into Zemax for optimization, and validating the design results through RMS radius evaluation of system performance.

After 80000 training iterations, the loss function decreases to 0.995 and stabilizes (Fig. 5). System performance testing with 100 randomly generated parameter combinations ($F_{\mathrm{h}\mathrm{a}\mathrm{l}\mathrm{f}\mathrm{?}X}$, $F_{\mathrm{h}\mathrm{a}\mathrm{l}\mathrm{f}\mathrm{?}Y}$, F) shows that 91 systems have an average RMS spot radius within 0?500 μm, with only 9 systems exceeding 500 μm in RMS radius. Among them, 3 systems have an average RMS spot radius approaching 2000 μm due to excessive position parameters (Fig. 6). Although these 91 systems may not directly qualify, they can be used as is or further optimized to improve the optical system structure. For example, with $F_{\mathrm{h}\mathrm{a}\mathrm{l}\mathrm{f}\mathrm{?}X}$=20, $F_{\mathrm{h}\mathrm{a}\mathrm{l}\mathrm{f}\mathrm{?}Y}$=15, and F=4, model prediction and post-optimization results are obtained (Fig. 7). After Zemax optimization, the system’s average RMS radius decreases from 30.34 to 22.67 μm, which is less than 1 arc minute after unit conversion, meeting human eye’s imaging standards (Figs. 8 and 9). Regarding modulation transfer function (MTF) performance, most fields maintain sagittal and meridional MTF values above 0.2 at 30 lp/mm spatial frequency (Fig. 10). The maximum grid distortion in the AR optical system’s perspective light path is controlled within 3.4%, meeting the human visual system’s tolerance for distortion (Fig. 12). The final AR optical system’s three-dimensional diagram is shown after adding the compensating prism (Fig. 11), and the overall performance of the final design meets AR display optical requirements.

The design method proposed in this paper combines optical simulation software with deep learning technology, enabling rapid generation of freeform prism optical structures within required spaces, followed by further optimization to enhance system image quality. This method significantly improves the design efficiency of freeform systems and reduces dependence on designer experience. The feasibility of the method is verified using 100 randomly generated parameter combinations ($F_{\mathrm{h}\mathrm{a}\mathrm{l}\mathrm{f}\mathrm{?}X}$, $F_{\mathrm{h}\mathrm{a}\mathrm{l}\mathrm{f}\mathrm{?}Y}$, F), as well as a specific case with $F_{\mathrm{h}\mathrm{a}\mathrm{l}\mathrm{f}\mathrm{?}X}$=20, $F_{\mathrm{h}\mathrm{a}\mathrm{l}\mathrm{f}\mathrm{?}Y}$=15, F=4, where the image quality meets human eye requirements. The method provides new insights into the rapid design and development of AR glasses optical systems. In addition, it can be effectively applied to other optical systems featuring freeform surfaces, offering vital technical support for AR-HMD applications in education, healthcare, industry, and military fields. The success of this method opens new possibilities for automated optical design in related fields such as virtual reality systems and advanced imaging devices. Its adaptability and scalability suggest that it can be extended to address other complex optical design challenges in various industrial and scientific fields. Future work may focus on refining the deep learning model, incorporating additional optical parameters, and exploring more complex optical system configurations to further expand the method’s applicability and effectiveness.