An efficient second-order nonlinear process with widely tunable pump bandwidth has always been a pursued goal, owing to extensive applications in wavelength division multiplexing networks,1 ultrashort pulse nonlinearity,2 quantum key distribution,3^–5 and broadband single-photon source generation.6^,7 In general, broad nonlinear bandwidth requires the phase-matching condition to be satisfied over a wide spectral range, which is equivalent to the simultaneous matching of both the group-velocity and phase-velocity of the interacting waves in the time domain.8^–10

Lithium niobate has been acknowledged as an excellent platform for nonlinear application. Highly efficient second-harmonic generation (SHG)11 and broadband SHG9 have been demonstrated on periodically poled lithium niobate (PPLN). In the last decade, thin-film lithium niobate (TFLN) has emerged as an ideal platform to manipulate and investigate the nonlinear interaction in the wavelength-scale,12^–16 where the compact optical structures could not only enable the huge enhancement of the conversion efficiency by tightly confining the optical field17^–19 but also provide degrees of freedom to tailor the group velocity, as well as the group-velocity dispersion through the structural geometry.20 As a result, broadband SHG under the waveguide configuration on TFLN has been extensively studied in recent years.21^–25 Using the quasi-phase matching (QPM) technique,26^,27 broadband SHG bandwidth over hundreds of nanometers has been demonstrated in the dispersion-engineered periodically poled ridged waveguide on TFLN,23 where the fundamental wave (FW) and second-harmonic (SH) wave have the same group velocity and stable overlap over the whole propagation distance, as the diagram in Fig. 1(a) has illustrated. Other methods that introduce the nonuniformity to broaden the bandwidth have also been reported,28^–30 while these conversion efficiencies are relatively low due to the limited interaction length.

In addition to direct group-velocity matching (GVM) by dispersion engineering or material nature, another alternative method of quasi-group-velocity matching (QGVM) has been used to achieve broadband nonlinear frequency conversion. This idea was first proposed by Fejer31 [see Fig. 1(b)], which introduced a wavelength-selected time-delay line (TDL) in periodically poled bulk LN32 and latter applied to fiber optics.33^–35 In this scheme, group velocities of interacting waves are no longer required to be always matched along the propagation direction, but rather the structure geometry is required to satisfy a certain ratio depending on the accumulated group-velocity mismatch. However, TDL-based devices typically suffer from a relatively large footprint, which puts forward an urgent need to realize the chip-scale integration for QGVM-based broadband nonlinearity.

Recently, anisotropic properties of the X-cut TFLN have been widely studied and applied to electro-optic modulation,36^,37 polarization manipulation,38^,39 sensing resolution enhancement,40 and poling-free QPM with the spontaneously inverted nonlinear coefficient.41^,42 To tailor the dispersion as well as the group velocity, such anisotropy also provides a potential path through birefringence-induced mode hybridization,43^,44 which can introduce the anomalous group-velocity mismatch (GVMM) and compensate for the temporal walk-off between FW and SH waves in a single waveguide, as Fig. 1(c) has illustrated. Based on the two routes, this paper demonstrates the greatly improved nonlinear bandwidth in the micronanophotonic structures on X-cut TFLN. Our experimental results show that broadband SHG can be found in a racetrack resonator or bent waveguide with a specific geometrical structure, in which the spontaneous quasi-phase-matching (SQPM) and QGVM are proven to be simultaneously satisfied in our simulation. Our study will open a new avenue toward broadband nonlinear functional devices on the anisotropic materials-based on-chip photonic platform.

A schematic of the typical structure proposed in this work is shown in Fig. 2(a). Here, a general SHG process is considered in a microracetrack resonator on X-cut TFLN, which has a straight section length of $L_0$ and a half-circle radius of $R$. The straight waveguide is aligned with the Y axis. As the TE-polarized FW and SH light circulate in the racetrack resonator, corresponding effective nonlinear coefficient $d_{\mathrm{eff}}$ varies with the direction of the wave vector $k$ in the relationship of $$d_{\mathrm{eff}}(\theta )=-d_{22}{\cos }^3\theta +d_{31}{\cos }^2\theta \sin \theta +d_{33}{\sin }^3\theta ,$$(1)where $\theta$ is the propagation azimuthal angle. From the above equation, we can see that $d_{\mathrm{eff}}$ oscillates between the $d_{33}$ and $-d_{33}$ in a $2\pi$-period of $\theta$, and it reaches the maximum and minimum values in two straight waveguides. The SHG process also depends on the phase mismatch between the FW and SH light. In the different sections of the racetrack resonator, the accumulated phase-mismatch can be calculated as follows: $$\mathrm{\Delta }{\varphi }_1=\mathrm{\Delta }{k}_1{L}_0,\mathrm{\Delta }{\varphi }_2={\int }_0^{\pi }\mathrm{\Delta }{k}_2(\theta )R\mathrm{d}\theta =\mathrm{\Delta }{k}_2^{\prime }\pi R,$$(2)where $\mathrm{\Delta }{k}_1$ and $\mathrm{\Delta }{k}_2(\theta )$ are the vector mismatch in the straight waveguide and the birefringent half-circle waveguide, respectively, which have the expression of the $\mathrm{\Delta }{k}_{1(2)}=4\pi \mathrm{\Delta }{n}_{1(2)}/\lambda$ with the FW wavelength $\lambda$ and the effective refractive index difference $\mathrm{\Delta }{n}_{1(2)}$ between the FW and SH. For simplicity, we use $\mathrm{\Delta }{k}_2^{\prime }$ as a reduced form of $\mathrm{\Delta }{k}_2(\theta )$ in the following analysis, which is defined by $\mathrm{\Delta }{k}_2^{\prime }=\frac{1}{\pi }{\int }_0^{\pi }\mathrm{\Delta }{k}_2(\theta )\mathrm{d}\theta$. Analogous to the common QPM process in the PPLN, the realization of the perfect SQPM requires $\mathrm{\Delta }{\varphi }_1=m\pi$ and $\mathrm{\Delta }{\varphi }_2=2N\pi$, where $m$ is an odd number and $N$ is an integer, corresponding to the QPM condition and continuation condition, respectively. In this case, the SH intensity will grow continuously in the poling-free racetrack resonator, as shown by the inset in Fig. 2(b). Furthermore, at the cost of some conversion efficiency, the above requirements can be further relaxed to the condition of $$\mathrm{\Delta }{\varphi }_1+\mathrm{\Delta }{\varphi }_2=(2N+m)\pi ,\mathrm{\Delta }{\varphi }_1\mathrm{mod}2\pi \ne 0,$$(3)which means that the slight shift from the $m\pi$ in the $\mathrm{\Delta }{\varphi }_1$ can be compensated by that in the $\mathrm{\Delta }{\varphi }_2$, so that an overall phase relationship between the FW and SH in a period remains unchanged. Compared to the other phase-matching methods, the conversion efficiency of the SQPM depends mainly on the structure size and is therefore usually lower than that of the common PPLN-based QPM.

Based on Eq. (3), we have the broadband SQPM condition, which requires at least the first-order derivative of $\mathrm{\Delta }{\varphi }_1+\mathrm{\Delta }{\varphi }_2$ to be equal to 0 at the central FW wavelength ${\lambda }_0$, which satisfies SQPM. Combining Eqs. (2) and (3), it can be derived to an equation related to the racetrack structure and dispersion, in the form of $$L_0(\frac{\text{d}\mathrm{\Delta }{k}_1}{\text{d}\lambda }{)}_{{\lambda }_0}=-\pi R(\frac{\text{d}\mathrm{\Delta }{k}_2^{\prime }}{\text{d}\lambda }{)}_{{\lambda }_0}.$$(4)

In general, it is difficult to fulfill Eq. (4) in an isotropic platform because the sign of $\text{d}\mathrm{\Delta }k/\text{d}\lambda$ is hard to change due to the relatively consistent mode transition between the straight and half-circle waveguides. However, for a birefringent racetrack resonator, the potential mode-hybridization in the half-circle waveguide could provide another degree of freedom to manipulate the modal dispersion. For example, in a ridge waveguide with a top width of $1\mu \mathrm{m}$ and a height of $0.38\mu \mathrm{m}$, the FW light with a wavelength of 1550 nm will maintain the single-mode condition during the intracavity propagation, while the SH light with the wavelength of 775 nm will experience a strong mode hybridization in the half-circle waveguide, as shown in Fig. 2(a). Therefore, its modal dispersion, as well as $\mathrm{d}\mathrm{\Delta }{k}_2^{\prime }/\mathrm{d}\lambda$ can be significantly changed compared to $\mathrm{d}\mathrm{\Delta }{k}_1/\mathrm{d}\lambda$ in the straight waveguide.

Specifically, we have simulated the effective refractive indices of all possible modes in the SH and FW bands in the half-circle waveguide on the $X$-cut TFLN. The conformal transformation was used to deal with the effect of the waveguide bending. In the SH band, the imported TE0 mode in the straight waveguide gradually transforms to the TM1 mode and then to the TE1 mode, as the propagation direction has been rotated by 90 deg in the quarter-circle waveguide. In Fig. 2(c), the width of the blue and red lines represents the change of indices in the FW wavelength range from 1530 to 1570 nm, from which it is obvious to see the different dispersion property at three stages of SH. The dashed line shows the $\theta$-dependent $\mathrm{d}\mathrm{\Delta }{k}_2/\mathrm{d}\lambda$ at the FW wavelength of 1550 nm in the half-circle waveguide. When both the FW and SH are in the TE0 modes, $\mathrm{d}\mathrm{\Delta }{k}_2/\mathrm{d}\lambda$ has a positive value as in the straight waveguide, while when the SH transfers to the higher-order modes, the sign of $\mathrm{d}\mathrm{\Delta }{k}_2/\mathrm{d}\lambda$ rapidly changes to negative. The average vector-mismatch dispersion in the straight and half-circle waveguides has been calculated versus the FW wavelength in Fig. 2(d), which shows an opposite sign over a broadband from 1520 to 1570 nm. From a time-domain point of view, the temporal walk-off accumulated in the straight waveguide between the interacting pulses in the FW and SH bands can be reduced or even compensated if the racetrack geometry satisfies Eq. (4), which means that these two pulses will overlap again when they enter the next straight waveguide.

To experimentally demonstrate the mode-hybridization-induced broadband SQPM SHG, we have first designed a dual-resonant 111th-order perfect SQPM racetrack resonator with a straight waveguide length $L_0$ of $245.4\mu \mathrm{m}$ and a half-circle radius $R$ of $129.03\mu \mathrm{m}$, corresponding to $\mathrm{\Delta }{\varphi }_1=111\pi$ and $\mathrm{\Delta }{\varphi }_2=2\times 91\pi$ at the FW wavelength of 1540 nm. Details of the design process can be found in Ref. 42. It is worth noting that the choice of geometric parameters is mainly determined by the fabrication process. By improving the machining process or using deep etching to suppress bending loss, the size of the microcavity can be further reduced if the above conditions are met. In this work, we focus on the dispersion property of this structure, as shown in Fig. 3(a). For the designed $R$ and $L_0$, $\mathrm{\Delta }{\varphi }_1$ decreases for the longer FW wavelength and $\mathrm{\Delta }{\varphi }_2$ vice versa, both of which have a close absolute value of the slope. Furthermore, we calculate the phase-mismatch summation in Fig. 3(b) and find it takes an extremum [$\text{d}(\mathrm{\Delta }{\varphi }_1+\mathrm{\Delta }{\varphi }_2)/\text{d}\lambda =0$] around the FW wavelength of 1560 nm, while the QPM condition ($\mathrm{\Delta }{\varphi }_1+\mathrm{\Delta }{\varphi }_2=293\pi$) is fulfilled at that of 1540 nm. By slightly tuning the temperature or the geometric parameter, the SQPM and QGVM conditions can be realized simultaneously. For example, as the temperature has risen by 10 K, and the $L_0$ has been reduced by 100 nm, the zero points of $\mathrm{\Delta }{\varphi }_1+\mathrm{\Delta }{\varphi }_2-293\pi$ and its first-order dispersion may shift and eventually meet at the FW wavelength of about 1552 nm. A detailed comparison is shown in Fig. 3(c). At the central FW wavelength of the perfect SQPM condition, $\mathrm{\Delta }{\varphi }_1$ is equal to $111\pi$, indicating a theoretically highest SH intensity; when it is shifted to $110.6\pi$ under the broadband SQPM condition, the overall SH intensity will be slightly lower. However, since the absolute change of $\mathrm{\Delta }{\varphi }_1$ is always within $\pi$, and in most cases less than $\pi /2$ as the FW wavelength is scanned from 1530 to 1570 nm, the conversion efficiency is not affected too much.

In our experiment, 111th-order perfect and phase-compensated SQPM racetrack resonators have been fabricated and characterized; the fabrication process can be found in Ref. 42. A pulley-type bus waveguide is used for on-chip coupling of the resonator, which has a center angle of 30 deg, top width of $0.8\mu \mathrm{m}$, and gap of $0.6\mu \mathrm{m}$, respectively. Figure 4(a) shows the experimental setup. A pair of grating couplers is used to couple the light in and out of the chip. The transmission spectrum of the FW light is recorded by an OSC under the small signal condition, and that of the shorter racetrack resonator is shown in Fig. 4(b). A typical TE0 mode around the wavelength of 1550 nm is zoomed in and fitted by the Lorentzian function, which shows an intrinsic Q factor of 155,000. Measured SH intensities in two resonators at different resonant modes of FW are shown in Fig. 4(d), from which a relatively large SHG spectrum from 1540 to 1565 nm can be observed with a 3-dB bandwidth of about 13 nm under phase-compensated SQPM. Compared with the SH bandwidth, we obtained in the perfect SQPM resonator, the bandwidth is significantly enhanced over 10-fold. For the broadband SHG, the highest on-chip SH power of about 100 nW is measured at the FW wavelength of 1555 nm under an on-chip pump power of 20 mW. The relatively low conversion efficiency can be attributed to the large SQPM order number used here. By exploring the lower-order SQPM (see the Supplementary Material), broad bandwidth and improved conversion efficiency can be achieved simultaneously. In addition, we find the SH intensity has a relatively large fluctuation with the FW wavelength from 1548 to 1558 nm, which can be attributed to the different coupling conditions and resonant mode mismatch.

Furthermore, we show that the mode-hybridization-induced QGVM can be combined with the SQPM in the bent waveguide configuration, as the schematic shown in Fig. 5(a). Compared with the SQPM-based racetrack resonator, the SQPM bent waveguide breaks the restriction of resonance, and thus can be applied to generate the SH light in a continuous broadband spectrum instead of a series of discrete modes. By selecting different hybrid modes in the SH band, we have designed the first-order SQPM bent waveguide with the QGVM satisfied at the same time; a detailed theoretical analysis can be found in the Supplementary Material. As shown in Fig. 5(b), here the half-circle radius $R$ is $75\mu \mathrm{m}$, and the straight section length $L_0$ is about $2.2\mu \mathrm{m}$, corresponding to the coherent length of the SHG process involved in TE0 modes at the FW wavelength of 1550 nm. The total length $L$ of the bent waveguide in the Z direction is about 15 mm, including 50 SQPM cycles. A fiber lens is used for coupling at the edge of the waveguide. The pump light for the FW is provided by a C-band femtosecond laser with a repetition frequency of 50 MHz and a pulse duration of about 500 fs. The spectrum of the pump light is shown in Fig. 5(c) with the red curve. In our experiment, with an on-chip average pump power of about 5.7 mW, a broadband upconversion spectrum at the 775 nm band is measured by the OSA at the output port as the blue curve shown in Fig. 5(c), which means that at least 16 nm 3 dB bandwidth has been achieved in the SQPM bent waveguide structure. According to the theoretical prediction, this bandwidth could be up to about 50 nm. It should be noted that the conversion efficiency of the bent waveguide in our experiment is mainly limited by the high insertion loss ($>50\mathrm{dB}$). By optimizing the structure parameters or improving the fabrication process, the conversion efficiency could be further increased.

In conclusion, we have demonstrated a new approach to achieve the QGVM SHG in the racetrack resonator and bent waveguide on X-cut TFLN. Based on the birefringence-induced mode transition of the SH light, the GVMM can flexibly change its sign from the half-circle to a straight waveguide during the light propagating for one cycle. SHG bandwidths of one and several 10-fold enhancements in the intracavity and bent waveguide have been achieved, which can be applied to other parametric processes, such as sum-frequency generation, difference-frequency generation, and optical parametric oscillation with the femtosecond laser pulse by further dispersion engineering and optimization of the structure. This work will significantly benefit chip-scale nonlinear frequency conversion between the ultrashort optical pulses and even the quantum states if the transmission loss is close to the intrinsic loss with the great progress in fabrication technology on a thin-film lithium niobate platform.

Tingge Yuan received her BS degree from Northwest University in 2019 and her PhD from Shanghai Jiao Tong University in 2024. Her research interests include nonlinear optics and optical microcavity on thin-film lithium niobate.

Jiangwei Wu is currently a PhD student at Shanghai Jiao Tong University. He received his BS degree from the School of Physics and Astronomy, Shanghai Jiao Tong University, in 2020. His research interests include integrated photonics and nonlinear optics on thin-film lithium niobate.

Xueyi Wang is currently a PhD student at Yale University. She received her MS degree from the School of Physics and Astronomy, Shanghai Jiao Tong University, in 2024, and her BS degree from the School of Physics, Xi’an Jiaotong University, in 2021. Her current research interests include nonlinear optics and integrated photonics.

Chengyu Chen is currently a PhD student at Shanghai Jiao Tong University. He received his honors bachelor’s degree from the Department of Physics, Huazhong University of Science and Technology, in 2021. His current research interests include optomechanics and optoelectronics.

Hao Li received his PhD from Shanghai Jiao Tong University in 2019. After two years of postdoctoral training at Shanghai Jiao Tong University, he joined the faculty of Shanghai Jiao Tong University in 2021, where he is currently an experimentalist. His current research interests include nanophotonics, nonlinear optics, integrated photonics, and quantum optics.

Bo Wang received his PhD from Peking University in 2017. After four-year postdoctoral research at Technion-Israel Institute of Technology, he joined Shanghai Jiao Tong University in 2021, where he is currently an associate professor. His research interests include nanophotonics, spin-orbit interactions of light, and photonic Ising machines.

Yuping Chen currently serves as a professor at the School of Physics and Astronomy, Shanghai Jiao Tong University (SJTU) in Shanghai, China. She received her PhD in optics from SJTU in 2002, and subsequently pursued a postdoctoral fellowship in Prof. R. W. Boyd’s group at the Institute of Optics, University of Rochester, starting in 2005. With over 100 published papers in refereed journals and more than 50 invited conference talks, her expertise lies in nonlinear and integrated optics. Her research focuses on on-chip nonlinear integrated photonic devices in thin-film lithium niobate and their applications.

Xianfeng Chen is a distinguished professor in the School of Physics and Astronomy at SJTU. He received his PhD in physics at SJTU in 1999. His current research focuses on nonlinear photonics, integrated photonics, biophotonics, and quantum information. In the past years, over 400 journal papers have been published in leading refereed journals and received more than 10,000 citations.