Optical neural networks (ONNs), including holographic neural networks^[1], large-scale photonic recurrent neural networks^[2], integrated photonic neural networks^[3], and diffractive neural networks (DNNs) at terahertz and near-infrared wavelengths^[4,5], have been an intense research field for deep learning. Specifically, ONNs use photons instead of electrons for computation, offering high-speed optical communication (speed of light in a medium) and massive parallelism of optical signals (multiplexing in time, space, wavelength, polarization, orbital angular momentum, etc.). Among all these ONNs, the free-space DNNs exploit the optical inference induced by the multilayer diffraction plates (amplitude or phase plates) and have demonstrated their outstanding advantages, such as high speed (the speed of light), lower power consumption, and high neuron density in applications including medical image analysis^[6,7], image recognition^[8–10], and classification^[11–13].

Recent progress in DNNs has demonstrated the requirement to fabricate DNNs with smaller device footprints^[5,14], pushing the working wavelengths from terahertz to the near-infrared region, then further to visible wavelengths, so that the compact DNN devices can be integrated with microscopic systems, realizing the DNNs for lab-on-chip applications^[15–17]. Additionally, the biological samples are commonly labeled by probes fluorescing at visible wavelengths^[18,19]. However, DNNs working at the visible wavelength fabricated by electron beam lithography (EBL) are limited to the two-dimensional (2D)^[20]. This large-sized DNN at the visible wavelength is not suitable for applications in microscopic imaging systems. Therefore, the development of three-dimensional (3D) micrometer-scale DNNs working at the visible wavelength is of great significance for applications in microscopic imaging systems.

This work presents the investigation of 3D micrometer-scale DNNs functioning at the visible wavelength manufactured by the two-photon nanolithography (TPN) technique^[21–26]. Two types of diffraction elements, i.e., cylindrical and cubical diffraction neurons, are studied for their amplitude and phase modulation properties, and finite-difference time-domain (FDTD) simulations demonstrated that cubical diffraction neurons provide an average of 50% higher optical amplitude and phase modulation efficiency compared with the cylindrical diffraction neurons. Based on the theoretical analysis of DNNs for handwritten digit classification^[20], the performance of micrometer-scale DNNs at the visible wavelength is further studied as a function of the sizes of the detecting area, and the number of diffraction layers for circularly distributed detector patterns. Experimentally, the single-layer and two-layer DNNs with 0.8μm×0.8μm cuboid diffraction neurons were fabricated using a home-built DLW system. The performance of the DNNs in handwritten digit classification was experimentally investigated, achieving an experimental image classification accuracy of 89.3% for the single-layer DNN, and 83.3% for the two-layer DNN. Our results paved the way for future applications of DNNs at visible wavelengths for potential integration with microscopic imaging systems, such as for biological imaging and cell recognition.

The schematic concept of DNN is shown in Fig. 1(A), where the DNN exploits the nature of light propagation in free space and the interaction of light fields (amplitude and phase) with thin diffraction layers to perform passive matrix multiplication operations in the optical domain. For DNNs performing image classification, the input image is projected to the first layer, where every neuron on the first layer transmits input light to other neurons on the following layers according to Huygens’ principle and the Rayleigh–Sommerfeld diffraction equation^[27]. A neuron of a given DNN layer can be considered as a secondary wave source that waves propagate in the following optical mode: wil=z−zir2(12πr+1jλ)ej2πrλ,(1)where l represents the lth layer of DNN, i represents the ith neuron with the coordinate (xi,yi,zi) in the lth layer, λ is the wavelength of input light and the working wavelength, r=(x−xi)2+(y−yi)2+(z−zi)2 represents the distance between the given neuron and the other neuron in the following layer, and j=−1.

Therefore, a neuron establishes connections with neurons in subsequent layers via a secondary wave that undergoes amplitude and phase modulation influenced by the input interference pattern from preceding layers and the neuron’s own modulation abilities. Defining several detector areas on the output layer of DNN, the classification criterion is to find the detector area with the strongest energy distribution.

To realize micrometer-scale DNNs at the visible wavelength, two typically used diffraction elements, i.e., cylindrical and cubical diffraction neurons, are studied for their amplitude and phase modulation properties^[5,28]. Theoretically, the optical properties of the cylindrical and cubical diffraction neurons can be understood, in the first approximation, within the Fresnel diffraction picture: the light transmitted through the neurons can be divided into two parts, one propagating in the neurons and the other in the surrounding medium (air). The FDTD method is used to further investigate the amplitude and phase modulation characteristics of both cylindrical and cubical neurons.

First, it is revealed that cubical diffraction neurons demonstrated a more effective phase modulation capability than cylindrical neurons. Compared with the optical path difference (OPD) formula (shown in Fig. 1(D) red dashed line), Δφ=2πh(n−n0)/λ, where h represents the height of neurons, n is the index of refraction of the photoresist, n0 is the index of refraction of air, and λ represents the wavelength of illuminated light, the phase modulation relationship of cubical neurons agrees highly with the OPD formula, while the cylindrical neurons demonstrate deviated phase modulation results compared to the OPD formula. This result is shown in Fig. 1(D).

In terms of amplitude modulation capability, cubical diffraction neurons demonstrate a higher amplitude modulation efficiency than cylindrical diffraction neurons. As is demonstrated in Fig. 1(E), cubical neurons maintain a higher amplitude modulation efficiency than cylindrical neurons, where the efficiency is defined as the ratio between the energy transmitted through the neuron and the energy leaked into the spacing between the neurons, which results from the excitation of high-order modes due to the spacings between cylindrical neurons^[5,29]. In our simulation, diffraction neurons with the size of 0.8 µm, and four diffraction neurons modulating the phase change from 0, π/4, π/2, and π are used to examine the amplitude modulation characteristics. In addition, the perfectly matched layer (PML) condition is applied as the boundary condition, and the phase and amplitude distribution data are obtained at the position of λ (532 nm).

Furthermore, it is found that the size of the detector area is an important factor influencing the performance of DNNs for image classification. First, we designed and investigated a single-layer DNN for two types of image classification. Given a single-layer DNN with a total size of 84 neurons by 84 neurons, the classification accuracy is a function of the size of the detector area, i.e., the number of neurons in the detector area. The numerical simulation results are shown in Fig. 2(A), where classification accuracy is plotted as a function of varied sizes of the detector area. With the decrease of the size of the detector area from 12neurons×12neurons to 6neurons×6neurons, the classification accuracy for single-layer DNN increases from 99.91% to 99.98%. The corresponding phase distribution of the single-layer DNN is plotted in Fig. 2(B).

In addition, the accuracy of a two-layer DNN for 10 types of image classification demonstrates a similar relationship in Fig. 2(C). It is easy to understand that the smaller detector area provides a smaller region for the previous layer for the convergence of energy, which leads to an area with more light energy, thus increasing the classification accuracy. The corresponding phase distribution of the two-layer DNN is plotted in Fig. 2(D).

This numerical simulation was performed by using Google’s TensorFlow (v2.7.0, Google Inc., USA) framework on a computer with a GeForce RTX 3060 graphical processing unit (GPU) and an Intel^R Core^TM i5-12490F central processing unit (CPU) @3.00 GHz and 32 GB of random access memory (RAM), running the Windows 11 operating system (Microsoft Corporation, USA). Phase-only modulation two-classifier DNN was designed by training a diffraction layer with 8090 images (720 validation images) from the Modified National Institute of Standards and Technology (MNIST) handwritten digit database^[30]. For the two-layer 10-classifier DNN, 42,000 training images and 28,000 validation images are also from the MNIST handwritten digit database. Both the single-layer DNN and two-layer DNN are simulated with a size of 67.2μm×67.2μm (84neurons×84neurons) under the wavelength of 532 nm, corresponding to a neuron size of 0.8 µm.

Based on the numerical simulation above, single-layer and two-layer DNNs functioning at the visible wavelength were experimentally fabricated using a home-built 3D DLW system. The system consists of a femtosecond laser at a wavelength of 532 nm and a 3D piezo nanotranslation stage (detailed system setup is shown in Section 4), where the arbitrary 3D microstructures can be fabricated through the accurate movement of the nanotranslation stage. To characterize the ability to classify and quantify the performance of single-layer DNN, we used the characterization setup depicted in Section 4. The input images of the handwritten digits were generated by spatially modulating the light from a 532 nm laser source using a spatial light modulator (SLM) and illuminating on the DNN using a 4f system and objective to resize the illuminated digit. The detector area on the output layer was imaged through an objective and detected using a charge-coupled device (CCD) camera.

The fabricated single-layer two-classifier DNN presents a good performance in the classification of 0 and 1. A CAD model of single-layer DNN in Fig. 3(A) realizes the process of the classification of numbers 0 and 1. Figure 3(B) shows the phase map of a single-layer two-classifier DNN and its scanning electron microscope (SEM) image. In Fig. 3(C), we report the characterization of single-layer two-classifier DNN illuminated by handwritten digits 0 and 1. For the 56 digits generated by the SLM, the experimental blind testing accuracy was 89.3%, close to the simulated result of 99.98%. To further evaluate the performance of the single-layer two-classifier DNN, we calculated and normalized the energy distribution of two detector areas. The normalized energy distribution of two detector areas of the experimental results conforms to the simulated results.

Furthermore, a DNN with ten detector areas and two layers was developed and investigated. The results are presented in Fig. 4. The CAD model of the two-layer 10-classifier DNN is depicted in Fig. 4(A), where the optimized parameters for the distance between the layers and the size of the neurons were determined through the iterative training, where the layer distance is 40 µm, diffraction layer size is 67.2μm×67.2μm, and the diffraction neuron is 0.8μm×0.8μm. Figure 4(B) displays the first layer and the second layer, which are phase maps generated by computer training using a subset of handwritten digits from the MNIST data set over 50 epochs. These phase maps were then converted into heights, representing the fabrication heights of the cuboid diffraction neurons in the 3D DLW. The fabricated results are illustrated in Figs. 4(C) and 4(D), and the zoom-in images demonstrate a high agreement between the corresponding height maps of the fabricated results and the simulation results. The two-layer DNN was fabricated in a single step, ensuring precise control over the distance between the layers and the relative position of the two diffraction layers.

The experimental characterization of a two-layer 10-classifier DNN was achieved using a benchtop optical setup, which is discussed in the previous section of this paper. Optical images of handwritten digits ranging from 0 to 9 were projected on the first layer of the two-layer 10-classifier DNN, the position of which was precisely aligned by a combination of a 3D microtranslation stage and a 2D microtranslation stage under an objective with a magnification of 40× and an NA of 0.75. The experimental outputs of the two-layer 10-classifier were captured using the CCD camera along the propagation direction of the optical images. The results are shown in Fig. 5, and the inset images of the handwritten digits are the experimental optical images captured on the surface of the sample without the two-layer 10-classifier. The sizes of these images are 100 µm by 100 µm.

As is shown in Fig. 5, the measured normalized intensity distributions demonstrate a consistent digit classification capability of the fabricated two-layer 10-classifier. For instance, in Fig. 5A(1), the optical image of digit 0 was incident onto the first layer of the DNN, and the diffraction energy distribution captured on the output position shows the strongest energy distribution at the detector area-0, while other detector areas have much less energy. This result is confirmed by both the energy distribution image and the histogram results in Fig. 5A(1). Therefore, the experimental results of all handwritten digits from 0 to 9 illustrated that the fabricated two-layer 10-classifier has an outstanding capability of image classification. For the 150 digits generated by the SLM, the experimental blind testing accuracy was 83.3%, close to the simulated result of 92.51%. The details of the theoretical calculation results and the experimental data, including the confusion matrix and the energy distribution, are shown in Section 4.

In conclusion, we designed and investigated the micrometer-scale DNNs at the visible wavelength. Different from DNNs working at the visible wavelength fabricated by EBL^[20], the 3D DLW technique was used in this work to fabricate an ultracompact micrometer-scale DNN. By utilizing the layer-by-layer slicing method to fabricate cubical neurons, the cross talk caused by the gap between adjacent neurons is mitigated. Theoretical investigations on the performance of micrometer-scale DNNs with both one and two diffraction layers are conducted, taking into account the size of the detecting areas and the number of diffraction layers. The single-layer DNN for the two-classifier demonstrated an experimental classification accuracy of 89.3% when the detector area measured 6neurons×6neurons, with a neuron size of 0.8 µm, with each layer consisting of 7056 neurons. Furthermore, the two-layer 10-classifier DNN achieved an accuracy of 83.3%, which highly agreed with the simulated classification accuracy of 92.51%. These findings highlight the potential of micrometer-scale DNNs in effectively classifying handwritten digits and cell recognition in biological microscopy. Due to the micrometer size of 3D micrometer-scale DNNs and the high classification accuracy in the experiments, it is possible to integrate the 3D micrometer-scale DNNs with microfluidic chips and microscopic imaging systems^[31,32] for high-efficiency cell counting^[33–35].

As shown in Fig. 6, a femtosecond fiber laser (Coherent Axon) provides laser light at a wavelength of 532 nm. The laser pulses with a width of less than 150 fs and a repetition rate of 80 MHz are steered into a 1.4 NA 100× oil immersion objective (Olympus). The process of fabrication can be captured by a CCD camera under the illumination of a red light-emitting diode (LED). The IP-Dip is used from Nanoscribe’s Dip in Laser Lithography (DiLL) technology by TPP^[36]. In the fabrication, high fabrication power and relatively low fabrication speed are utilized to fabricate our micrometer-scale DNN. Therefore, the microstructure can maintain the structural integrity to a large degree.

The characterization system is shown in Fig. 7(A). The input images of handwritten digits are generated by the SLM using a laser source at the wavelength of 532 nm, and the experimental results are captured by a CCD camera. The classification results are shown in Fig. 7(B), where the vertical axis represents the classification results of the DNN, and the horizontal axis represents the true labels of input digits. The classification accuracy achieved 89.3% for the single-layer two-classifier DNN and 83.3% for the two-layer 10-classifier DNN. The test data are not used in the training of the DNNs.

To accurately align the sample and the input images, a border-only images (shown in the figure below) with the same size as the test images are loaded onto the SLM, where the size of the border-only image is adjusted to adapt to the size of the DNN sample. This process enables the accurate generation of test images with the same size of the sample. At the same time, we used a 3D micrometer translation stage to align the sample with image at the focusing plane of the objective. Therefore, these methods ensure that in the x–y plane, the sample and the tested images can be aligned with an accuracy of less than 0.5 µm under the objective. For the z direction, after the alignment in the x–y plane, the sample moves slightly away from focusing plane using the 3D micrometer translation stage; thereafter, the output intensity of the DNN can be obtained by the CCD camera.