To meet the demands of our information-driven society, the rapid growth of data traffic has necessitated significant capacity expansion in fiber-optic communications^[1]. Space-division multiplexing (SDM), which explores the degree of freedom in the transverse spatial domain, has emerged as a promising solution for high-capacity communications^[2]. Mode-division multiplexing (MDM), a subset of SDM, exploits diverse spatial modes to establish multiple information channels that carry independent data streams synchronously, thereby significantly enhancing data capacity and spectral efficiency in a single fiber^[3]. In MDM fiber-optic communication systems, several mode basis sets are applicable for constructing parallel information channels, including linearly polarized (LP) modes^[4,5], vector modes^[6,7], and orbital angular momentum (OAM) modes^[8,9]. Among these, the LP mode basis is widely adopted in practical applications due to the relative ease with which these modes can be excited and detected^[10]. It is crucial to extract specific information about the spatial characteristics of LP modes transmitted through a single fiber, which can be extended to mode visualization for managing multiple mode channels. Furthermore, based on the scalable mode channels in the transverse spatial domain, spatial characteristics of different modes can be utilized to achieve higher-capacity and higher-density information encoding and decoding, which can be further extended to applications in high-complexity and high-security encryption and decryption. To achieve mode visualization and system miniaturization, it is essential to employ ultra-compact devices with low-complexity designs that facilitate integrated manipulation and intelligent recognition of modes transmitted in a fiber.

Metasurfaces^[11], planar optics patterned with subwavelength-scale nanoscatterers, have demonstrated the ability to manipulate various light properties^[12], including amplitude^[13], phase^[14], polarization^[15], and frequency^[16]. Metasurfaces have achieved significant achievements in multifunctional optical components across multiple dimensions, such as wavelength^[17], polarization^[18], and spatial mode^[19]. With their ultra-compact construction, wide-range integration, and multi-dimension function, metasurfaces provide a novel platform for MDM, offering both theoretical exploration and practical application. For mode emission, metasurfaces have been used to generate multiple spatial modes, either in free space or integrated on fiber ports, including OAM modes with varying topological charges^[20], higher-order LP modes^[21], and arbitrarily structured light^[22]. In the realm of mode division, current metasurface-based devices have successfully realized the multiplexing and demultiplexing of LP modes in two orthogonal polarizations^[23] and OAM modes with different topological charges^[24]. Furthermore, practical applications have seen the emergence of OAM holography, allowing the multiplexing of a broad range of OAM modes, leading to OAM-encoded holograms for high-security optical encryption^[25]. Beam steering systems that combine control of polarization and wavelength have facilitated the scaling of information channels for high-capacity optical communication^[26–28]. However, the recognition and visualization of independent spatial modes using a platform that integrates metasurface devices with optical fibers remains a significant challenge.

In this work, we propose an intelligent mode-visualizing metasurface for recognizing LP modes transmitted in a single fiber by integrating neural network techniques and metasurfaces into an MDM platform. Specifically, the metasurface functions as a Fourier grating, enabling the switching of two holographic images in their respective target spatial regions by altering the incident mode, as shown in Fig. 1. This capability allows for mode recognition and visualization by displaying distinct characters in the captured images on the far-field observation plane. Our metasurface employs nanostructures with identical geometric size and different orientation angles to achieve phase modulation based on the geometric phase. To enhance computation speed and ensure operation accuracy, we incorporate neural network algorithms into the optimization process for the phase profile of the metasurface. A series of experiments was conducted to validate our framework, where the holographic images for the two modes were accurately displayed in their designated target regions. The ultra-compact construction applicability and multi-dimension manipulation capability of our metasurface, combined with the high-speed computation and large-volume operation of the neural network architecture, offer significant advantages, including device compatibility, design flexibility, and function scalability for intelligent mode management. Looking ahead, our framework paves a practical pathway across a wide range of applications, including intelligent metasurface-driven pattern recognition and object classification, information encoding and decoding, as well as encryption and decryption.

We use a neural network architecture to optimize the phase profile of the metasurface. Figure 2 provides an illustration of the training architecture. Initially, we collect input datasets containing amplitudes A01 and A11, phases P01 and P11, of standard and transformed mode optical fields, which are derived from planar transformations in four dimensions, containing translations along x and y axes, rotations along the central axis of mode fields, and magnifications around the central point of mode fields (See Appendix A for details about data collection).

To visualize the two modes, we create two target images respectively with characters “LP01” and “LP11” positioned at different spatial regions, the intensities of which represent the designed output targets, denoted as T01 and T11. These target images undergo preprocessing, including distortion correction, energy compensation, and value normalization. By the phase profile of the metasurface φ and fast Fourier transform (FFT), the actual outputs O01 and O11 can be computed as $$\begin{gathered} O01=|FFT[A01exp(iP01+iφ)]|2, \\ O11=|FFT[A11exp(iP11+iφ)]|2. \end{gathered}$$(1)

We customize the loss function Loss, which is computed as $$\begin{gathered} Loss=αSUM[(O01−T01)2×WMask01] \\ +βSUM[(O11−T11)2×WMask11]. \end{gathered}$$(2)

The terms WMask01 and WMask11 refer to the “weight masks” applied to T01 and T11, which are binary masks to emphasize the target output areas and reduce the influence of background regions, thereby enhancing the convergence speed of the training network. The squared difference between the designed and actual outputs is weighted by the masks and then aggregated using the SUM operator. The coefficients α and β are, respectively, set as 1.15 and 1 for energy compensation between the two-mode optical fields. Subsequently, the Loss is delivered in the backpropagation to the neural network, and the gradient is updated by the Adam optimizer to generate the new phase profile (See Appendix B for details about the training configuration).

To achieve high transmittance and precise manipulation at the operation wavelength of 1550 nm, we carefully design the metasurface unit cells, focusing on the geometry dimensions and orientation angles.

The unit-cell nanostructure, made from monocrystalline silicon, consists of a nanobrick sitting on a substrate, as shown in Fig. 3(a). The structural parameters include cell size C, length L, width W, height H, and orientation angle α of the nanobrick. We used COMSOL Multiphysics software to scan cell size C, length L, and width W of the nanobrick. The final nanostructure design, with dimensions of C=860nm, L=680nm, W=260nm, and H=1000nm, achieves a high transmittance over 70% and complete phase modulation from 0 to 2π for the cross-polarized component, as shown in Fig. 3(b).

The optimized phase profile of the metasurface, consisting of 1000 pixel × 1000 pixel, is illustrated in Fig. 3(c), and an enlarged part (20 pixel × 20 pixel) at the upper-left corner of the phase profile is shown in Fig. 3(d). Subsequently, the orientation angles α of the nanobricks are derived from the optimized phase profile φ of the metasurface, following the principle of geometric phase, which is expressed as φ=2α.(3)

As a result, single-sized nanostructures with identical geometric dimensions but varying orientation angles are arranged in a single-layer configuration, accomplishing the structural construction of the metasurface.

To evaluate the theoretical feasibility of our framework, we compute the Kirchhoff diffractions using the constructed metasurface. Input optical fields of the LP01 and LP11 modes, which are normally projected onto the metasurface, are shown in Figs. 4(a) and 4(e). After modulations by the metasurface, the simulated output holographic images, observed at 10 cm from the metasurface, are, respectively, shown in Figs. 4(b) and 4(f). The simulated target images of both modes as depicted by the white boxes are accurately positioned in their respective target regions, which are extracted and enlarged displaying the characters “LP01” and “LP11,” as shown in Figs. 4(c) and 4(g), respectively. The designed target images used in the training architecture are depicted in Figs. 4(d) and 4(h). These evidences indicate a high degree of agreement between simulated and designed target images. The simulation results indicate that our constructed metasurface enables the switching of the holographic images under the conversion of the incident spatial modes.

To demonstrate the functional applicability of our framework, we fabricate a metasurface sample and set up an optical system to characterize mode recognition and visualization (Appendix C for details about sample fabrication). Figure 5 presents scanning electron microscope (SEM) images of the fabricated metasurface at different magnifications, in which the nanobrick adhesion appears, caused by fabrication imperfections. This adhesion reduces the holographic efficiency and negatively impacts the experimental performance. This issue can be mitigated through improvements in both the design and fabrication processes. From the design perspective, increasing the dimensional contrast between the nanobrick cell sizes and lengths can reduce the probability of adhesion. In terms of fabrication, decreasing the length and width of the nanobricks can help correct size deviations.

The optical system used to capture the holographic images is illustrated in Fig. 6(a). A tunable laser (THORLABS TLX1) served as the light source, and a few-mode fiber supported the LP01 and LP11 modes, operating at a wavelength of 1550 nm. The mode fields were collimated by a lens (THORLABS PAF2-2C) and then passed through a circular polarizer (THORLABS CP1L1550) to adjust their polarization states. The mode fields projected onto the metasurface were captured by an infrared camera (HAMAMATSU C12741-03), as shown in Figs. 6(b) and 6(e). The holographic images were focused by a lens (LATEK MBCX10606-C) and then received by an infrared detector board (LATEK IRDC1-200S-M-SP230614) with a black cardstock to block zero-order light. The holographic images were captured using a commercial camera (Nikon D5100). As shown in Figs. 6(c) and 6(f), the patterns of the holographic images transformed when the incident modes switched between LP01 and LP11 modes. The target regions in these holographic images are extracted and enlarged to display the details, revealing the characters “LP01” and “LP11,” which showed the spatial characteristics of the incident modes, as shown in Figs. 6(d) and 6(g).

Distortions observed in the holographic images primarily arise from aberrations induced by the high numerical aperture focusing lens. Additionally, structural imperfections in the fabricated metasurface and spatial misalignments between the incident mode field and the metasurface further degrade image quality. Integrating the metasurface directly onto the fiber end face can reduce these spatial misalignments, thereby enhancing the accuracy and fidelity of the experimental results.

We employed the Pearson correlation coefficient to evaluate the mode fidelity, which is calculated as ρ(A,B)=cov(A,B)σAσB.(4)Here, cov(A,B) represents the covariance between A and B, and σA and σB denote the standard deviations of A and B. We calculated the Pearson correlation coefficients between the experimental target images without being focused and the corresponding designed target images, obtaining values of 0.80 and 0.78, respectively, for the LP01 and LP11 modes, indicating the high fidelity of the experimental images. In comparison, the Pearson correlation coefficients between the simulated target images and the designed target images were calculated, as shown in Figs. 7(a) and 7(b). Moreover, the energy ratio between the target image of the incident mode and that of the other mode in the simulated and experimental holographic image was calculated to evaluate the mode crosstalk, as shown in Figs. 7(c) and 7(d). Although the crosstalk increased and the fidelity declined, target characters for each mode in the holographic images were identified in experiments. Overall, the experimental results demonstrate that our fabricated metasurface successfully achieved the recognition and visualization of two spatial modes through the patterns of the holographic images.

In summary, we theoretically designed and experimentally demonstrated an intelligent mode-visualizing metasurface that recognizes fiber modes via holographic imaging. This single-layer, single-cell approach integrates metasurface-based phase modulation with neural network optimization, ensuring image projection across multiple information channels. By combining the ultra-compact structure and concise design of metasurfaces with the high-speed computation and scalability of neural networks, our framework enhances compatibility, flexibility, and expandability for mode management. These attributes make it highly promising for object classification, information encoding and decoding, and encryption and decryption.