Lithium-niobate-on-insulator (LNOI) devices have developed rapidly in the past decades, benefitting from fast-growing fabrication technologies^[1–4]. Its excellent electro-optical and nonlinear optical features have promised bottleneck breakthroughs in electro-optical modulators^[5–8], integrated lasers^[9–11], nonlinear wavelength converters^[12–14], quantum sources^[15–18], etc., and may lead to revolutions in optical communication, microwave photonics, and quantum integration applications. Toward this, one of the remaining unsolved tasks is to design and fabricate efficient edge couplers for fiber-chip coupling and packaging for plug-and-play applications, with mode-selective excitation. Previously, several kinds of high-efficiency edge couplers have already been developed by means of adiabatic evolution^[19–24]. However, they mainly focus on excitation of the first-order fundamental mode and lack flexibility for on-demand mode-selective excitations, including those of different polarization patterns, mode numbers, wavelengths, etc.

To obtain on-demand coupling of a certain mode, the exciting and required modes should be phase-matched and have non-zero spatial overlap at the cross-section plane. Different from traditional photonic integrated circuits (PICs), the mode field diameter (MFD) of the strong confined LNOI waveguide is only around 1μm2, which differs a lot from fibers. To achieve higher coupling efficiency, usually a gradual LNOI waveguide with an MFD gradient that changed from the normal waveguide size to near-zero and a low-refractive-index cladding mode with an MFD matching that of fibers is necessary for adiabatic evolution design^[23]. Such sub-wavelength structures increase the fabrication complexity and constrict the overall feasibility of the edge coupler to be phase-matched with various waveguide modes, as limited by mode hybridization^[25]. Thus, usually only the fundamental mode of the LNOI waveguide can be excited directly, and additional on-chip mode converters^[26–28] based on etched-LN structures have to be used for transversing the fundamental mode to the desired modes, which limits the flexibility of multi-mode applications and potential promotion of large-scale integrations.

Here, we design a new type of edge coupler with direct mode-selective excitation ability, using SiON cladding grating structures. A SiON cladding layer with periodic structures is introduced on a uniform LNOI waveguide for high-efficient butt-coupling to fibers and direct excitation of multiple core waveguide modes simultaneously. A cladding waveguide is fabricated on the edge of the cladding layer to provide a guided mode with an MFD that matches that of lens fibers for high-efficiency butt-coupling. Then periodic structures are introduced into the cladding layer to compensate for the momentum mismatch between the co-propagating guided mode and the desired LNOI waveguide mode so that the desired waveguide mode can be excited directly with high efficiency. Specifically, we provide a typical example of a Z-cut LNOI waveguide with a thickness of 600 nm, an etch depth of 350 nm, and a width of 1.4 µm. At the launch of the TM cladding mode, TM00, TE00, and TE10 core waveguide modes can be excited, respectively, at optimized period and grating length with a tunable central wavelength. The period of the SiON gratings is calculated to be all over 2 µm, which can be processed by mature i-line UV exposure. The results provide a compact and low-cost mode-selective edge coupler for the LNOI platform, which may be beneficial for large-scale multiport PICs.

The scheme of the edge coupler is shown in Fig. 1. It consists of a SiON cladding layer with a grating structure above the LNOI ridge waveguide and the outermost SiO2 protecting layer. Our design is based on a Z-cut LNOI wafer, with a 600-nm-thick LiNbO3 thin film bonded on a 2-µm-thick thermally grown SiO2 layer above a Si substrate. In order to reduce the propagation loss, we assume that 350 nm is etched to form the ridge waveguide, with a top width of 1.4 µm and a sidewall angle of 60°. The refractive index of SiON (∼1.56) is larger than that of SiO2 (∼1.44), hence supporting the cladding mode in the SiON layer. The size of the SiON cross-section is designed to be 2.7μm×3μm so that the cladding mode in SiON possesses a similar MFD (∼2.8μm) with lensed fiber (∼2.0μm), which allows efficient butt-coupling. In the SiON grating region, the reciprocal vector compensates for the wave vector mismatch between the cladding mode and the core mode so that the cladding mode can be efficiently coupled to the core mode during propagation.

To realize efficient coupling between the cladding mode in SiON and the core mode in LiNbO3, the grating is used to compensate for their momentum mismatch. The grating period Λ is calculated as^[29]Λ=λneff,core−neff,cladding,(1)where neff,core and neff,cladding are effective indices of the core mode in the LiNbO3 layer and the cladding mode in SiON, respectively. We choose the fundamental TM mode in the SiON cladding as the input mode in all the simulations below. In the waveguide we use here, we discuss three core modes TM00, TE00, and TE10, and the grating periods for mode coupling between the cladding mode and these modes under different wavelengths are shown in Fig. 2(a). Aside from momentum conservation, the modal overlap also remarkably affects the coupling between modes. The field distributions of the three core modes and the TM cladding mode are shown in Fig. 2(b). It is noted that the TM cladding mode has horizontal symmetry in Ez and anti-symmetry in Ey. TM00 and TE10 core modes have the same symmetrical characteristics as TM cladding mode, and the mode overlap is non-zero between these core modes and the cladding mode. So the conversion from TM cladding mode to TM00 and TE10 core modes is supported directly in this structure. Meanwhile, TE00 has the opposite horizontal symmetry of the TM cladding mode, implying near-zero overlap. It means that coupling between the TE00 core mode and the TM cladding mode hardly occurs in this scheme. However, such mismatches can be dealt with using unsymmetric gratings, which we will discuss later.

The conversion from TM cladding mode to TM00 core mode simulated by the eigenmode expansion (EME) solver is shown in Fig. 3. Here we input the TM cladding mode at the right-side surface of the cladding layer as the light source, as shown in Fig. 1. For the central wavelength of 1550 nm, the grating period is optimized to be 4.041 µm. Because the phase-mismatch between the two modes is compensated exactly by the grating, the energy is coupled between the TM cladding mode and TM00 core mode periodically along propagation [Fig. 3(a)]. The maximum conversion occurs at the position L=18Λ, with a conversion efficiency of 82%. Such coupling efficiency is comparable to other edge couplers^[23,30–32]. Since it only requires a micrometers structure, it can hence be fabricated via low-cost i-line UV exposure technology. The conversion spectrum in Fig. 3(b) shows the 3 dB bandwidth of 42.0 nm. Note that the effective index of TE10 is close to that of TM00 mode, and a slight conversion to TE10 mode can be seen simultaneously in this case. However, at the grating period of 4.041 µm, TM00 possesses the most efficient conversion at 1550 nm, while TE10 conversion occurs at around 1592 nm. Since the region of TE10 conversion is far from that of TM00 coupling in the wavelength, it has a limited influence on the conversion efficiency of TM00. As a result, the mode suppression ratio (MSR) of TM00 to TE10 is 20.2 dB at 1550 nm. We also simulate the conversion efficiency evolution of TM00 core mode at different wavelengths to further characterize its conversion bandwidth, as shown in Fig. 3(c). The results show a good performance in the first grating period. We also simulate the mode evaluation from TE cladding to TE00 to study the polarization sensitivity of this coupler. As shown in Fig. 3(b), the conversion efficiency is very low at wavelengths from 1400 to 1700 nm, showing that our coupler is polarization-sensitive.

Using the same structure, the conversion from TM cladding mode to TE10 core mode is simulated similarly, and the results are shown in Fig. 4. For efficient coupling at the 1550 nm, the grating period is optimized to be 3.875 µm. Figure 4(a) depicts the periodic energy conversion from TM cladding mode to TE10 core mode, and the maximum conversion occurs at the position L=75Λ, with a conversion efficiency of 61%. It is noted that TM00 conversion also happens simultaneously in this case. The 3 dB bandwidth of TE10 conversion is 11.3 nm as shown in Fig. 4(b). At the period of 3.875 µm, TE10 possesses the most efficient conversion at 1550 nm, while TM00 conversion occurs at a wavelength near 1515 nm, which is in agreement with the phase-matching condition in Fig. 2(a). It is noted that TM00 has a much broader conversion bandwidth than TE10 because of the stronger modal overlap and, thus, easy coupling between TM00 core mode and TM cladding mode. Such a feature also leads to the mixture of TM00 in TE10. As a result, the MSR between TE10 and TM00 is 8.91 dB at 1550 nm, as shown in Fig. 4(b). This MSR can be improved by increasing the propagation length, where the TE10 peak near L=225Λ has a higher MSR of 10.2 dB. This result demonstrates that, with a proper grating period, the long-period grating coupler can convert the cladding mode to a high-order core mode in even perpendicular polarization on demand, showing the mode-excitation flexibility of such a coupler.

Then we study the wavelength tunability of the edge coupler. The conversion efficiency of different wavelengths from the TM cladding mode to TM00 and TE10 core modes at three typical periods are simulated and shown in Fig. 5. When the period is changed from 3.823 to 4.248 µm, the central wavelength of TM00 conversion is tuned from 1500 to 1600 nm, demonstrating that our design can cover the whole S-C-L band. For TE10 mode, with the grating period changed from 3.662 to 4.084 µm, we can cover the wavelength from 1500 to 1600 nm as well [Fig. 5(b)]. The results demonstrate a flexible tunability of the resonant wavelengths.

Furthermore, as mentioned above, TE00 core mode has the opposite symmetrical characteristic of the TM cladding mode, which prohibits the coupling between the TE00 core mode and the TM cladding mode. To enhance the coupling between them, we broke the symmetry of the grating structure. Herein an asymmetric grating is proposed to realize TE00 conversion as shown in Fig. 6. In this scheme, the SiON is etched at either the left or right side in turns period by period, which leads to non-zero overlap between the TM cladding mode and the TE00 core mode, and allows conversion between the two modes.

For the central wavelength of 1550 nm, the grating period in the structure is optimized to be 3.047 µm, and the energy conversion of TE00 mode along propagation is shown in Fig. 7(a). The maximum conversion occurs at the position L=635Λ, with a conversion efficiency of 85%. The 3 dB bandwidth of the conversion spectrum is 1.55 nm, as shown in Fig. 7(b). Because the effective index of TE00 is far from those of TE10 or TM00 modes, energy is hardly coupled to the TE10 or TM00 mode. The MSR of TE00 to TE10 is 35.2 dB at 1550 nm, which implies the high purity of TE00 mode conversion.

For this asymmetric cladding grating coupler, we also study its wavelength tunability. As shown in Fig. 8, when the period is changed from 2.922 to 3.172 µm, the central wavelength of TE00 conversion is tuned from 1500 to 1600 nm, which also covers the whole S-C-L band.

In conclusion, we have proposed a new type of edge coupler for LNOI devices using SiON cladding grating structures to realize efficient fiber-chip coupling and selectively excite distinct spatial modes simultaneously. For a typical example of C-band coupling here, the needed grating period is calculated to be over 2 µm, which is feasible with i-line UV lithography. At the launch of the TM cladding mode, TM00 waveguide mode can be excited with maximum coupling efficiency exceeding 85%, and the maximum 3 dB bandwidth reaches 42 nm. With further optimization via a chirp grating structure or dispersion-engineered waveguide, the coupler bandwidth can be further increased. The excitation of high-order TE10 waveguide mode can also be realized directly using the same structure with optimized parameters. By breaking the symmetry of the coupler’s grating, the conversion of TM cladding mode to TE00 waveguide mode is achieved with high efficiency, enabling direct polarization-rotating during the fiber-chip coupling. Compared to on-chip mode converters, our coupler can achieve efficient fiber-chip coupling and mode excitation to the specific core mode simultaneously using a single device; hence, it is a more compact choice for mode-selected excitation from fiber to chip. Furthermore, by controlling the grating period, the central wavelength of the excited modes can be easily tuned to cover the whole S-C-L band. Our results provide a low-cost coupler for LNOI PICs with spatial mode selectivity, which will be helpful for plug-and-play applications in optical communication, microwave photonics, and quantum integration areas.