Hetero-integrated perovskite/Si3N4 on-chip photonic system

Mar. 30 , 2025SJ_Zhang

Abstract

Integrated photonic chips hold substantial potential in optical communications, computing, light detection and ranging, sensing, and imaging, offering exceptional data throughput and low power consumption. A key objective is to build a monolithic on-chip photonic system that integrates light sources, processors and photodetectors on a single chip. However, this remains challenging due to limitations in materials engineering, chip integration techniques and design methods. Perovskites offer simple fabrication, tolerance to lattice mismatch, flexible bandgap tunability and low cost, making them promising for hetero-integration with silicon photonics. Here we propose and experimentally realize a near-infrared monolithic on-chip photonic system based on a perovskite/silicon nitride photonic platform, developing nano-hetero-integration technology to integrate efficient light-emitting diodes, high-performance processors and sensitive photodetectors. Photonic neural networks are implemented to perform photonic simulations and computer vision tasks. Our network efficiently predicts the topological invariant in a two-dimensional disordered Su–Schrieffer–Heeger model and simulates nonlinear topological models with an average fidelity of 87%. In addition, we achieve a test accuracy of over 85% in edge detection and 56% on the CIFAR-10 dataset using a scaled-up architecture. This work addresses the challenge of integrating diverse nanophotonic components on a chip, offering a promising solution for chip-integrated multifunctional photonic information processing.

Main

With the rapid development of information technology, there is an urgent need for information-processing chips that offer high data rates, low loss, minimal distortion and crosstalk, and reduced power consumption. Photonic chips, which use photons to carry information, are promising candidates^1,2. Their high integration and robust data-processing capabilities are crucial for advanced applications. A long-term goal is to realize a monolithic on-chip photonic system in which light sources, signal processors and photodetectors are hetero-integrated on a single chip. Notable efforts have been focused on improving the performance of individual light-emitting diodes (LEDs)^3,4,5,6 and photodetectors^7,8,9,10. Advancements in materials engineering and micro- and nanofabrication techniques have facilitated the integration of these two types of component, enabling optical interconnects and communication systems¹¹. This includes the integration of LEDs with photodetectors^12,13,14,15, LEDs with waveguides^16,17,18 and photodetectors with waveguides^19,20,21,22.

Various schemes based on different material platforms and integration technologies have been proposed to achieve monolithic photonic chips. III–V semiconductors are ideal for both light source and photodetector. Photonic chips that integrate these elements with waveguides have facilitated a range of functions including signal transmission²³, on-chip light generation and detection²⁴, high-speed data transmission²⁵, real-time full-duplex communication²⁶ and chemical sensing²⁷. By utilizing various III–V semiconductor components, the operational wavelength of these photonic chips can be adjusted from the ultraviolet to mid-infrared regimes. Germanium, a group-IV semiconductor, is another excellent candidate for both light source and photodetector applications in optical communications. In 2021, an on-chip optical interconnection using a germanium LED, silicon waveguide and germanium photodetector was demonstrated²⁸. Additionally, semiconducting single-walled carbon nanotubes can function as both light sources and photodetectors for optical communications. In 2017, an electrically driven monolithic plasmonic circuit was demonstrated for data transport, featuring a carbon nanotube surface plasmon polariton source, a gold (Au)-strip waveguide and a carbon nanotube photovoltaic detector²⁹. However, without an on-chip processor, these photonic circuits with single-waveguide structures remain constrained in their capacity for complex information processing. A comparison of different schemes of monolithic photonic integration is summarized in Extended Data Table 1. In particular, previously reported schemes face notable technical challenges, including managing material heterogeneity, addressing lattice mismatch, optimizing device performance and improving the optical coupling efficiency. Achieving a high-performance, stable and reliable on-chip photonic system still remains challenging due to stringent compatibility requirements across materials engineering, integration techniques and design methods needed to integrate light emission, processing and detection components.

As an emerging complement to conventional inorganic semiconductor materials, perovskites offer several unique advantages for hetero-integration with silicon photonics platforms. First, perovskites enable simple and low-temperature fabrication. Unlike III–V semiconductors, which require high-temperature, complex epitaxial growth processes that induce thermal stress and increase costs^30,31, perovskites can be solution processed at low temperatures, reducing stress and improving compatibility with silicon platforms^32,33,34. Second, perovskites exhibit a high tolerance to lattice mismatch. Although conventional inorganic materials often suffer from defect formation due to lattice mismatch with the silicon platform^35,36, perovskites demonstrate both high defect tolerance and self-healing properties, potentially enhancing device reliability^37,38. Third, perovskites offer excellent bandgap tunability (Supplementary Fig. 8), achieved through simple compositional tuning^39,40, covering a wide spectral range without notable phase separation or spectral broadening issues commonly associated with III–V materials^41,42. Finally, perovskites stand out due to their low cost and material abundance, making them highly suitable for widespread deployment^43,44.

Here we report an on-chip photonic system based on a heterogeneously integrated metal halide perovskite/silicon nitride (Si₃N₄) photonic platform, which incorporates a high-performance on-chip LED light source with an external quantum efficiency (EQE) of 21.2% and a 5.3 MHz cutoff frequency (at 4.5 V), two on-chip sensitive photodetectors with an 8.3 pA dark current at −0.15 V and a rapid response time of 44 ns, and a 13-layer Si₃N₄ locally connected photonic neural network for photonic computing (Fig. 1a). When monolithically integrated, the system achieves a 3 dB cutoff frequency (f_{3 dB}) of 2.2 MHz. The on-chip photonic system leverages the capabilities of the hetero-integrated perovskite/Si₃N₄ platform to establish multifunctional photonic neural networks for photonic simulations (Fig. 1b) and computer vision tasks (Fig. 1c).

Fig. 1: Proposal of the monolithic hetero-integration of perovskite/Si₃N₄ on-chip photonic system.

a, Schematic of the monolithic hetero-integration of perovskite/Si₃N₄ on-chip photonic system. The system comprised an efficient and stable LED, a 13-layer (L₁–L₁₃) locally connected photonic network with 15 inputs and 2 outputs for photonic computing, and 2 highly sensitive and fast-response photodetectors. NL, nonlinear layer. b,c, Schematic illustrating photonic simulations (b) and computer vision tasks (c) performed using the proposed photonic system.

Two topological models with ill-defined energy bands, demanding substantial computational resources, are selected to showcase the capabilities of the photonic neural network in photonic simulations with reduced computational costs. Adaptive learning ability is first validated through the efficient predictions of the topological invariant in a two-dimensional (2D) disordered Su–Schrieffer–Heeger (SSH) model. Building on this foundation, the capabilities of the network are further enhanced to handle photonic simulations in the SSH model with third-order nonlinearity, achieving an impressive average fidelity of 87%. We then extend its application to computer vision tasks, highlighting the network’s adaptability across diverse problem-solving scenarios. It achieves a test accuracy of over 85% on the Canny Edge Detection dataset and a notable accuracy of 56% on the more challenging CIFAR-10 classification task using a scaled-up architecture. This dual success in both photonic simulations and computer vision tasks underscores the versatility of the network and marks a important step forward in the development of monolithically integrated intelligent photonic chips. This work overcomes a long-standing challenge of integrating different classes of photonic components on a single chip, opening new avenues for exploring more fundamental physical problems and potentially revolutionizing various technological domains.

Results and discussion

Monolithically integrated perovskite/Si₃N₄ photonic platform

We propose a strategy for monolithically integrating perovskite LEDs, Si₃N₄ photonic processors and perovskite photodetectors on a single chip. To realize both light emission and detection on a photonic chip, we develop high-performance metal halide perovskite materials with considerable overlap and continuous tunability between absorption and emission (Supplementary Fig. 8). Leveraging micro- and nanofabrication techniques, we achieve the heterogeneous integration of perovskite and Si₃N₄ waveguide networks (Supplementary Fig. 1). The system consists of an LED with a feature size of 1 mm × 2 mm; a 13-layer locally connected photonic network for photonic computing with 15 inputs and 2 outputs (Supplementary Fig. 2); and 2 photodetectors, each with a feature size of 0.5 mm × 0.5 mm.

The device structure of the perovskite diode is glass/indium tin oxide (ITO)/polyethyleneimine ethoxylated (PEIE)-modified zinc oxide (ZnO)/perovskite/poly(9,9-dioctylfluorene-co-N-(4-butylphenyl) diphenylamine)/molybdenum oxide (MoO₃)/Au (Methods). For the on-chip devices, glass/chromium (Cr)/Au/Si₃N₄ waveguides/silicon oxide were fabricated under the ITO film. The electroluminescence (EL) spectrum and absorption curve of the perovskite materials show a considerable degree of overlap, with a relatively small Stokes shift (Supplementary Figs. 6a and 8). This overlap indicates that a considerable portion of the emitted light can be reabsorbed by the same material, suggesting that perovskite diodes can effectively function in both LED and photodetector modes. Top view, cross-sectional view and energy-dispersive X-ray spectroscopy images illustrate the submicrometre structures in the perovskite films, consistent with previous studies^45,46 (Supplementary Figs. 6b,c and 7).

We first demonstrate the performance of perovskite diodes in the LED mode. As shown in Fig. 2a, under forward bias, electron and hole injection into the perovskite layer generate EL. Due to the high quality of the perovskite films, our perovskite LEDs exhibit high performance independent of size (Supplementary Fig. 9). An optimized perovskite LED with an area of 4 mm² shows a peak EQE of 22.8% and peak radiance of 166 W sr⁻¹ m⁻² (Supplementary Fig. 9e). Moreover, the 2 mm² perovskite LED used in the photonic system shows a peak EQE of 21.2% and peak radiance of ~105 W sr⁻¹ m⁻² with low operating voltages, high reproducibility and high spectral stability (Fig. 2b and Supplementary Figs. 10 and 11). The temporal response is critical for on-chip-integrated devices^11,47. To assess this, we further measure the transient EL of the perovskite LEDs at varying frequencies (Supplementary Fig. 13). As shown in Fig. 2c, the f_{3 dB} value increases with the drive voltage due to the improved carrier injection^12,48,49,50, reaching a maximum of 5.3 MHz at 4.5 V (Supplementary Fig. 12). The longer photoluminescence lifetime (Supplementary Fig. 12d) and lower current density of perovskites limits their bandwidth compared with III–V inorganic materials^51,52.

Fig. 2: Characterizations of the monolithic hetero-integrated perovskite/Si₃N₄ on-chip photonic system.

a, Schematic of a perovskite diode working in the LED mode. PeLED, perovskite LED. The inset shows an image of the electrically driven perovskite LED with an area of 1 mm × 2 mm. E_c, conduction band; E_v, valence band; E_Fn, electron quasi-Fermi level; E_Fp, hole quasi-Fermi level. b, EQE and radiance versus current density curves of the optimized perovskite LEDs. c, Frequency response of the perovskite LEDs with different bias voltages. d, Schematic of the perovskite diode working in the photodetector mode. PePD, perovskite photodetector. The inset shows the image of the perovskite photodetector with an area of 0.5 mm × 0.5 mm. e, Current–voltage (I–V) curves of the perovskite photodetector under dark and under illuminated light for different light powers. The peak wavelength of the excitation light source is 808 nm, closely matching the electroluminescent peak wavelength of the perovskite. f, TPC characteristics under zero bias of perovskite photodetectors with different device areas. a.u., arbitrary units. g, Schematic of the perovskite LED, photodetector and Si₃N₄ waveguide on a single chip. The inset shows an image of the monolithically integrated on-chip photonic system. h, I–V curves for different structures. J_PD, output current density of the on-chip perovskite photodetector. i, Frequency response of the on-chip devices with different structures.

Then, we evaluate the performance of the perovskite diodes in the photodetector mode. Under zero bias, the diodes are excited by photons with energies above the perovskite bandgap, generating photocarriers that are separated and extracted by the charge transport layers and electrodes (Fig. 2d). The EQE spectra of the photodetectors exhibit a broad spectral response ranging from the ultraviolet to near-infrared regimes (300–800 nm) (Supplementary Fig. 14a), with a high responsivity of up to 0.134 A W⁻¹ at 640 nm under zero bias, ranking among the best-performing perovskite photodetectors^53,54 (Supplementary Fig. 14b). Under zero bias, the photodetectors maintain 95% of their performance compared with operation at a −1 V bias voltage, indicating efficient performance under zero bias. The detection efficiency of the photodetector gradually decreases with increasing bias voltage, as explained by the energy band diagrams (Supplementary Note 3 and Supplementary Fig. 14d–f). Our photodetectors exhibit low dark currents, down to 8.3 pA at −0.15 V under 365 nm excitation, broad spectral response and consistent high sensitivity (Fig. 2e and Supplementary Figs. 15a, 16 and 17a). On the basis of the responsivity and noise current, we obtain the noise equivalent power and specific detectivity (D*) under zero bias, which are 2.2 × 10⁻¹² W Hz^−0.5 and 2.2 × 10¹¹ Jones at 800 nm (Supplementary Fig. 15). Transient photocurrent (TPC) measurements are conducted to further assess the temporal response of our photodetectors (Fig. 2f). The response time is obtained by fitting the TPC curves with an exponential function. As the active area decreases from 0.81 mm² to 0.2 mm², the device responds more rapidly, showing a reduction in the response time from 129 ns to 29 ns (Supplementary Fig. 17c). However, this reduction in area also lowers the signal-to-noise ratio (Supplementary Fig. 17d). The response speed is primarily limited by parasitic capacitance and series resistance within the photodetectors and the measurement circuit^12,54,55.

After evaluating the individual LEDs and photodetectors, we investigated the overall performance of the monolithic hetero-integrated perovskite/Si₃N₄ photonic system. Figure 2g shows the diagram of the integrated LED, Si₃N₄ waveguides network and photodetectors on a single chip. The perovskite LED emits light into the grating coupler, which guides its propagation through the waveguides, achieving an EQE of 0.34% when accounting for the coupling efficiency. The light is then transmitted through the waveguide and coupled to the photodetector, where it dissociates into carriers and is collected by the electrodes. Considering the grating coupling efficiency, the waveguide-integrated photodetector exhibits a responsivity of 4 mA W^–1 at 798 nm under zero bias (Supplementary Notes 1 and 4 and Supplementary Fig. 3). To demonstrate the performance of the monolithic integration system, we compare four structures: RGW, RG, R and None (Supplementary Notes 1 and 2 and Supplementary Figs. 3, 4, 18 and 19). The RGW structure demonstrates a notably higher output current compared with the lowest output current in the R structure, highlighting the advantages of our monolithic hetero-integrated structural design (Fig. 2h). Bandwidth measurements further validate the superior performance of our structure design (Fig. 2i and Supplementary Fig. 20), with the RGW structure achieving a maximum f_{3 dB} value of 2.2 MHz, compared with only 0.55 MHz for the R structure. The integrated on-chip system exhibits a lower bandwidth than a single perovskite LED due to optical losses in the photonic system.

Topological invariant calculations in a 2D disordered SSH model

Here we demonstrate the capabilities of our on-chip photonic system for intelligent photonic computing using photonic neural networks to calculate topological invariants in a 2D disordered SSH model. Disorder, introduced by randomly displacing sites, disrupts the spatial translational symmetry, leading to an undefined band structure due to the absence of periodicity. Such a disorder is known to induce Anderson localization^56,57, halting wave transport and can trigger topological phase transitions⁵⁸. However, the absence of translational symmetry complicates the calculation of energy spectrum eigenstates and the topological invariants of the system, posing substantial computational challenges. Using neural networks for solving such a system greatly reduces both computational workload and processing time.

Figure 3a depicts the schematic of a 2D disordered SSH model in which disorder is introduced as positional fluctuations in the original lattice, equivalent to disorder-induced variations in the coupling coefficients. In this context, we did not include potential sub-nearest-neighbour couplings, a common simplification in tight-bonding models. In this model, the averaged mean chiral displacement (AMCD)^59,60,61 serves as the topological invariant (Supplementary Note 5). We calculated the temporal evolution of AMCD for crystal cell numbers of N = 4 and N = 20. In the SSH model without disorder, the result for the AMCD converges to half the winding number of the energy band as time approaches infinity. Despite the presence of disorder, AMCD effectively characterizes the topological properties of the system. For N = 20, with fixed intercell coupling coefficients at ν₀ = 1, and intracell coupling coefficients set at u₀ = 0.5 (topological) and u₀ = 2 (trivial), disorder introduced in these coefficients follows a uniform distribution with u ~ U(u₀, 0.5) and ν ~ U(ν₀, 0.5). We compute the evolution operators for 1,000 randomly generated sets of variables to average the AMCDs (Fig. 3b). In particular, even when the initial parameters are topological, the average AMCD slightly falls below 0.5, indicating that disorder impacts but does not break the topological non-trivial properties of the system. For different values of N, we use different distributions to calculate the AMCD (Supplementary Fig. 21). For N = 4, disorder via normally distributed coupling coefficients (u ~ N(u₀, σ_u), ν ~ N(ν₀, σ_ν), σ_u = σ_ν = 0.5) induces topological properties in the trivial phase, demonstrating disorder-induced topological phase transition (Fig. 3c). Eigenenergies near zero for topological and trivial phases under both open-boundary condition (OBC) and periodic-boundary condition (PBC) are shown in Fig. 3d–g. Under topological conditions, a bandgap opens in the eigenvalue spectrum with PBC, whereas eigenstates appear within this gap under OBC, indicating the presence of topological states. In trivial cases, there are no such eigenstates observed. As depicted in Fig. 3h–k, certain components of the edge and corner states appear in the bulk or at the boundaries due to Anderson localization caused by disorder, leading to eigenmodes nearly degenerate with the topological states.

Fig. 3: AMCD calculations in a 2D disordered SSH model.

a, Schematic of the 2D disordered SSH model. b,c, Calculated results of the AMCD evolutions for N = 20 (b) and N = 4 (c). d–g, Calculated eigenenergies for both topological (d and f) and trivial (e and g) phases under OBC and PBC. h–k, Calculated topological states. l,m, Large (l) and small (m) network architectures for AMCD calculations. n–q, Training loss and test results for large (n and o) and small (p and q) AMCD networks. r, Experimental results of a small-scale network for AMCD calculations.

We apply photonic neural networks to predict AMCD values in the 2D disordered SSH model, characterizing the topological phase. We compare two photonic neural network architectures: a comprehensive theoretical model and a scaled-down experimental version, both featuring a unit structure of a 15-input, 2-output photonic neural network. These models are simulated in PyTorch for gradient computation in optimization⁶² (Supplementary Note 7 provides the implementation and training details). The larger-scale model for AMCD calculation features a distinctive architecture comprising multiple unit arrays (Fig. 3l). The nonlinearity of the model comes from the saturated absorption in Bi₂Te₃ waveguides between layers 5 and 6 (Supplementary Fig. 5). Despite the limited power variation of the nonlinear layer, it plays a crucial role in ensuring the optimal performance of the photonic neural network, particularly for more complex tasks (Supplementary Note 8 and Supplementary Figs. 24–28). A flexible four-layer structure with 35 units handles a 78-channel input for a 2-channel output. For a practical demonstration, we downsized the larger model to address an AMCD problem with smaller cell numbers, replicating the final two layers with eight units (Fig. 3m).

We conducted four training rounds with a unique initialization to assess the network’s learning capabilities. Training loss curves for both large and small models are depicted in Fig. 3n–p, showing consistent behaviour with similar final losses at 40,000 iterations for the large model and 4,000 iterations for the small model. Test results from the first training round (Fig. 3o–q) demonstrate the precision of the networks in predicting the 2-channel AMCD values (namely, AMCD_x and AMCD_y). Figure 3m shows the phase shift values for each node, representing the trained weights from the first training round. We tested the trained weights on 24 randomly selected test samples for experimental validation (Fig. 3r). The experimental results aligned with the theoretical predictions, effectively bridging the gap between theoretical design and application, as well as demonstrating the potential of photonic neural networks in solving complex disordered topological problems.

Photonic simulations in nonlinear topological model

In the disordered model, the energy band is ill-defined due to the broken spatial translational symmetry, escalating computational complexity. Similarly, introducing nonlinearity into a topological model complicates the precise energy band definition, presenting considerable challenges in computational complexity. These challenges are mitigated by leveraging neural networks, which reduce the computational demands by benefiting from the powerful learning capabilities. Here we introduce third-order nonlinear coefficients in the SSH model to control topological phase transitions^63,64, thereby incorporating the Kerr effect to manipulate light propagation (Supplementary Note 6).

First, we consider a trivial-phase SSH model without an edge state, setting the intracell coupling coefficient to 2 and the intercell coupling coefficient to 1. We calculate the state evolution under a nonlinear model. Figure 4a illustrates the theoretical calculation of the wave-function distribution at varying incident intensities for T = 64, showing bulk distribution at low incident intensity and edge localization as the intensity increases. The spatial distribution at the boundaries (Fig. 4b) further supports this observation. Lacking an inherent edge state in the trivial SSH model, the observed edge distribution results from a nonlinear-induced localization. Nonlinear interactions introduce on-site potentials and reduce the effective coupling between the nearest sites. When light is incident at the outermost edge of the system (Fig. 4e), it shows minimal diffusion towards the bulk, remaining confined to edge propagation as the input intensity increases. We also investigated the system’s response to a Gaussian envelope input. As shown in Fig. 4c,d,f, the increasing intensity balances the diffusion (equivalent to spatial diffraction) and nonlinearity-induced self-focusing, stabilizing the propagation of spatial solitons. The self-defocusing phenomenon is shown in Supplementary Fig. 22.

Fig. 4: Time-dependent photonic simulations in the nonlinear topological model.

a, Wave-function distribution at T = 64 with an edge input in the nonlinear SSH model. b, Proportion of wave functions across ten lattice points at the edge with changing input intensities. c, Wave-function distribution with a Gaussian envelope in the middle of the system at T = 64. d, Proportion of wave functions across ten lattice points at the centre with changing input intensities. e, Wave-function evolution transitions from bulk to edge with an increasing input intensity. f, Wave-function evolution with a Gaussian beam input, forming spatial solitons as the input intensity increases. g, Theoretical network architecture for time-dependent nonlinear photonic simulations in the SSH model. h, Training fidelity curve with colour-coded time steps. i, Fidelity distribution across the test dataset. j, Average fidelity across all the 64 time steps for the test dataset.

Here we expand the photonic neural network capabilities for this kind of nonlinear photonic simulations, leveraging an advanced network design to predict state evolution from the given incident states. The architecture for the nonlinear SSH model (Fig. 4g) resembles that used for AMCD calculations, featuring arrays of trapezoidal unit structures scaled to handle 64-channel inputs and outputs (a total of 864 units). This model directly uses randomly generated complex wave-function values as inputs, and the target outputs are the state evolution with time computed using an evolution operator. The training objective is to maximize fidelity, a measure of the network’s precision in replicating the desired wave functions. Figure 4h illustrates the fidelity curve over 64 different time steps for varying input intensities, where the colour code from red to purple represents the time variables. Each curve maintains consistent initial conditions and sample sequencing. The limited expressive capacity results in fluctuating fidelity values over time t, averaging 87% fidelity during training. Figure 4i highlights the fidelity distribution at critical time steps: 1, 22, 43 and 64, whereas Fig. 4j details the average fidelity across all the time steps in the test dataset, also with an average fidelity of 87%. This high averaged fidelity underscores the network’s effectiveness in this task. Consistent performance in training and testing indicates the robustness of our network against overfitting.

Computer vision tasks

Next, we demonstrate the capabilities of our network in computer vision, confirming its ability to generalize across diverse visual datasets. We selected the CIFAR-10 dataset⁶⁵, a complex collection of real-world RGB images, as a challenging test case. For experimental verification, we trained a single network unit as a basic, trainable edge detector. The integration of these tasks highlights the network’s adaptability and potential for widespread applications in various fields of computer vision.

The approach to edge detection uses a training dataset of three images from the ImageNet dataset⁶⁶, segmented into 3 × 3 patches. These patches are encoded from RGB to a complex amplitude, where hue represents the phase and luminosity indicates the intensity. Training targets, generated using Canny Edge Detection⁶⁷, label patches based on the edge status of the centre pixel. For illustrative purposes, Fig. 5a demonstrates this process through our custom schematic. We use a weighted sampling strategy during training to offset the imbalance of non-edge versus edge instances, which is not used when calculating confusion matrices. The training loss and accuracy curves indicate 80% accuracy on the training set (Fig. 5c,d). The test evaluation, shown in the confusion matrices (Fig. 5e), yields similar results. Figure 5b displays the trained network weights from the first training round, used for subsequent experimental measurements. Experimental analysis on 50 edge and 50 non-edge patches from the test dataset confirmed the capabilities of the network, as evidenced by the measured confusion matrix (Fig. 5f) and output distribution (Fig. 5g). The elevated experimental accuracy is attributed to the relatively small sample size.

Fig. 5: Edge detection and image classification tasks.

a, Schematic of 3 × 3 image patches encoded into complex amplitudes in the training dataset. b, Network architecture for edge detection and trained phase distribution. c,d, Training loss (c) and training accuracy (d) curves for edge detection over four training rounds with different initializations. e, Test confusion matrices of edge detection. f, Experimental results of the confusion matrix for edge detection, based on 100 randomly selected test dataset samples (50 edge cases and 50 non-edge cases). g, Normalized network output distribution, represented as logits for edge/non-edge classification. h, Network designed for CIFAR-10 classification. i, Task for CIFAR-10 classification. j,k, Training loss (j) and training accuracy (k) curves for CIFAR-10 classification over four individual training rounds with different initializations. l, Test confusion matrix of CIFAR-10 from the first training round.

Our network, designed for the CIFAR-10 classification task (Fig. 5h), utilizes 917 unit structures. The architecture mirrors that of modern computer vision networks, integrating feature extraction, feature processing and a classification head for a 10-channel output, where the highest intensity indicates the predicted class. It processes 32 × 32 RGB images (Fig. 5i), encoded similar to edge detection inputs. Training loss and accuracy curves over four separate training rounds with different initializations are shown in Fig. 5j,k. The network achieves 60% accuracy on the training dataset, and the confusion matrix for the first training round on the CIFAR-10 test dataset (Fig. 5l) demonstrates a test accuracy of 56%. The results are worthwhile given the modest number of trainable parameters (95,368). Without weight sharing, each parameter in our network represents a total number of 95,368 floating-point operations. By contrast, a basic convolutional neural network like LeNet-5 (ref. ⁶⁸), used for the MNIST dataset⁶⁹ that achieves a 98.5% accuracy, requires 61,706 parameters and 833,040 floating-point operations. The training of our network on the MNIST dataset reaches a test dataset accuracy of 97.1% (Supplementary Fig. 23). This efficiency and ability to process images on this scale with an optical neural network are unprecedented, highlighting the advanced capacity of our design and the potential of these networks in the realm of computer vision.

Conclusion

In this work, we report the realization of an on-chip photonic system utilizing hetero-integrated perovskite/Si₃N₄ platform and demonstrate its multifunctionality. Our system features a high-performance perovskite LED, two highly sensitive photodetectors and a 13-layer Si₃N₄ locally connected network for on-chip photonic computing. Multifunctional photonic neural networks are implemented for photonic simulations and computer vision tasks. These applications include calculating the topological invariant in a 2D disordered SSH model both theoretically and experimentally, conducting photonic simulations in the nonlinear SSH model with an average fidelity of 87%, performing edge detection with a measured test accuracy of over 85%, as well as classifying the CIFAR-10 task in the scaled-up architecture with a test accuracy of 56%. The energy consumption and computing speed are estimated to be ~100 fJ and ~2.2 Mbit s^–1, respectively (~2 Gbit s^–1 if utilizing bandwidth to accelerate computing; Supplementary Note 9 and Supplementary Fig. 29). This work addresses a long-term challenge of integrating diverse essential nanophotonic components on a single chip, laying the foundation for future research and development in the field of integrated intelligent photonic chips.

Methods

Materials

Poly(9,9-dioctylfluorene-co-N-(4-butylphenyl) diphenylamine) (average molecular weight, ~50,000 g mol⁻¹) was purchased from American Dye Source. Chlorobenzene (extra dry, 99.8%), ethanol (extra dry, 99.5%), N,N-dimethylformamide (99.5%) and ethyl acetate (high-performance liquid chromatography (HPLC) grade) were purchased from J&K Chemical. Formamidinium iodide (FAI, 99.9%) and MoO₃ (99.9%) were purchased from Xi’an Polymer Light Technology. Caesium bromide (99.99%) and lead bromide (PbI₂, 99.999%) were purchased from Sigma-Aldrich. Beta-alaninamide hydrochloride (BAH, >98%) was purchased from Tokyo Chemical Industry (TCI). All the materials were used as received without further purification.

Fabrication of Si₃N₄ waveguide network

First, a bilayer comprising 100 nm/10 nm of Au/Cr was deposited on a quartz substrate using thermal evaporation (Kurt J. Lesker Labline PVD 75), serving as the bottom reflector/adhesion layer. Subsequently, a 300 nm Si₃N₄ film was grown on this substrate using plasma-enhanced chemical vapour deposition (Oxford Instruments PlasmaPro 100). The desired pattern of the waveguide structure was then written on the 450-nm-thick negative resist (AR-N) using electron-beam lithography (JEOL JBX-9500FS). Following this, in combination with reactive ion beam etching (Oxford PlasmaPro 100 reactive ion beam etching system), the Si₃N₄ waveguide network was fabricated, incorporating input–output grating couplers. Subsequently, a longitudinal window was created between the fifth and sixth layers of the waveguide network using a positive resist (ZEP520A). Pulsed laser deposition technology (SKY Technology Development PLD10) combined with a XeCl (λ = 308 nm) excimer laser (Coherent COMPex 205) was then used to deposit a 30-nm-thick film of Bi₂Te₃ at room temperature on the network coated with a resist pattern. Then, we removed the resist by using the lift-off process. Thus, a nonlinear activation layer was prepared. Afterwards, a 1 μm silicon oxide cladding layer was deposited on the Si₃N₄ waveguide network using plasma-enhanced chemical vapour deposition (Oxford Instruments PlasmaPro 100). Subsequently, a 150 nm ITO film was sputtered as the bottom electrode using a magnetron sputtering process (MSP-3200).

Fabrication of near-infrared perovskite diodes

The perovskite precursor solution comprising 0.013 mmol CsI, 0.247 mmol FAI, 0.13 mmol PbI₂ and 0.048 mmol BAH was added into 1 ml N,N-dimethylformamide. The perovskite solution was filtered with 0.22 μm filters before spin coating. Colloidal ZnO nanoparticles were spin coated onto the ITO-coated glass substrates at 5,000 r.p.m. for 45 s and annealed in air at 150 °C for 10 min. Next, a PEIE solution (0.03 wt% in isopropanol) was spin coated onto the ZnO surface at 5,000 r.p.m. for 45 s followed by annealing at 100 °C for 10 min. Then, the substrates were transferred into a nitrogen-filled glovebox. The perovskite films were prepared by spin coating the precursor solution onto the PEIE-treated ZnO films at 5,000 r.p.m. for 70 s, followed by annealing at 96 °C for 10 min. Poly(9,9-dioctylfluorene-co-N-(4-butylphenyl) diphenylamine) in chlorobenzene (12 mg ml⁻¹) was spin coated at 4,000 r.p.m. for 45 s. Finally, the MoO₃/Au electrodes were deposited using a thermal evaporation system through a shadow mask under a base pressure of 4 × 10⁻⁴ Pa. All the devices were encapsulated with ultraviolet epoxy (NOA81, Thorlabs)/cover glass before subsequent measurements.

Fabrication of bandgap-tunable perovskite films

The mixed halide precursor solution was prepared by dissolving stoichiometric amounts of FAI, FABr, CsI, PbBr₂ and PbI₂ (molar ratio (FAI + FABr + CsI):(PbBr₂ + PbI₂) = 2:1) in N,N-dimethylformamide, with a Pb²⁺ concentration of 0.13 mol l⁻¹. BAH was added as an additive at a concentration of 6 mg ml⁻¹. To adjust the bandgap, the ratio of the components was modified according to the stoichiometric requirements. All the precursor solutions were stirred at room temperature in a nitrogen-filled glovebox for 2 h and filtered through 0.22 μm polytetrafluoroethylene filters before use. After cleaning, the substrates were exposed to ultraviolet–ozone for 15 min. The synthesized ZnO nanoparticles were spin coated onto the substrates at 5,000 r.p.m. for 45 s, followed by annealing at 150 °C for 10 min. PEIE (0.04 wt% in isopropanol) was then spin coated onto the ZnO layer at 5,000 r.p.m. for 45 s and annealed at 100 °C for 10 min. After cooling, the substrates were transferred to a nitrogen-filled glovebox. To fabricate the perovskite films, the precursors were spin coated onto the PEIE-modified ZnO substrates at 5,000 r.p.m. for 60 s, followed by annealing at 100 °C for 10 min. The prepared perovskite thin films were placed in an integrating sphere and excited by a 405 nm laser to emit photoluminescence, which was collected by an Ocean Optics QEPro spectrometer coupled with a fibre.

Ultraviolet–visible absorption measurements

Ultraviolet–visible absorption spectra were obtained with a Cary 7000 ultraviolet–visible–near-infrared spectrophotometer at a scan rate of 300 nm min⁻¹. The near-infrared perovskite films used in the measurements were deposited on glass/ITO/PEIE-modified ZnO.

Time-correlated single-photon counting measurements of a perovskite film

The photoluminescence intensity and lifetime were obtained using a time-correlated single-photon counting setup (PicoQuant, MicroTime 200). A 400 nm, 500 kHz pulsed laser was focused onto the sample with a ×50 objective.

Characterization of near-infrared perovskite diodes working in the LED mode

The current density (J)–voltage (V)–luminance (L) characteristics of the perovskite LEDs were measured using a Keithley 2400 sourcemeter unit and a calibrated photometer system consisting of an integrating sphere that was fibre coupled with an Ocean Optics QEPro spectrometer in a nitrogen-filled glovebox. The setup was calibrated using a standard visible–near-infrared light source (HL-3P-INT-CAL Plus, Ocean Optics). The J–V characteristics of the devices were scanned with an increasing rate of 0.1 V s⁻¹. Stability measurements of perovskite LEDs were performed in a nitrogen-filled glovebox at ambient temperature (20 ± 5 °C) using a multichannel LED lifetime-testing system (Crysco). The concentrations of water and oxygen in the glovebox were maintained to be <0.01 ppm.

Characterization of near-infrared perovskite diodes working in the photodetector mode

The photo-to-current EQE of the devices under various bias voltages were measured using a solar cell spectral response measurement system (QE-R, EnliTech), which was calibrated with a standard Si crystalline solar cell. For photocurrent I–V curve measurements, 365 nm, 780 nm and 808 nm LEDs were used to illuminate the devices. The photocurrent was measured using a Keithley 2400 sourcemeter. The irradiance on the perovskite photodetector is adjusted by varying the attenuator in front of the excitation light source. For the dark J–V curve measurements, the perovskite photodetectors were placed in a black sealed metal box. The noise current of the devices was measured using the Keithley 4200 device. For the operational stability test of the perovskite photodetector, a commercialized LED was used to excite the active area. The output current on the sample resistor was recorded by an Ocean Optics QEPro spectrometer.

Characterization of the integrated on-chip photonic system

For the integrated on-chip measurement, the perovskite LEDs were driven by a Keithley 2400 sourcemeter unit, and the photodetector was connected to a 1 MΩ sample resistor. The partial voltage across the sample resistor was collected by another Keithley 2400 sourcemeter. For the bandwidth measurement, the perovskite LED on the integrated on-chip devices was driven by a DG1062Z function generator, and the partial voltage across the sample resistor was collected by a light power detector (Newport, 818-UV/DB). When characterizing the transient EL curves, all of the cables that connected the perovskite LEDs and oscilloscope needed to be as short as possible; the perovskite LEDs, the sample resistor and the oscilloscope were connected with a fast Bayonet Neill–Concelman connector to minimize the influence of the inductance of the circuit.

Bandwidth measurement based on a visible-light communication system

The bandwidth of the perovskite LEDs was measured using a visible-light communication system. This system consists of a vector network analyser, a d.c. sourcemeter, a lens and an avalanche photodiode (APD). A radio-frequency signal from the S1 port of the vector network analyser (300 kHz to 20 GHz) and a d.c. bias from the d.c. sourcemeter (Keithley 2450) were superimposed into a radio-frequency d.c. signal by a biasing device (Mini-Circuits, ZFBT-4R2GW-FT+, 0.1–4,200 MHz) and applied to the perovskite LEDs. The d.c. bias ranged from 2.0 V to 4.5 V. The modulated light emitted by the perovskite LEDs was focused by a lens and detected by the APD (APD110A2/M). An optical fibre with a diameter of 1,600 μm was used to couple the light between the lens and the APD. The signal from the APD was fed into the S2 port of the vector network analyser, which determined the operating bandwidth of the perovskite LEDs based on the S21 parameter. Before the measurements, the vector network analyser was calibrated for frequency range, ports, loads, short circuits and open circuits.

Transient EL and bandwidth measurements of near-infrared perovskite LEDs

Short, square voltage pulses were generated by a DG1062Z function generator and applied to the perovskite LEDs. A 50 Ω resistor was placed in series with the perovskite LEDs. Transient voltages were measured using a four-channel oscilloscope (Tektronix oscilloscope MDO34) and the transient current was calculated from the voltage across the resistor in series with the perovskite LEDs. The bandwidth of the perovskite LEDs was driven by a square voltage and the average EL intensities were measured using an integrating sphere connected to a light power detector (Newport, 818-UV/DB), with the perovskite LEDs place inside the integrating sphere. When characterizing the transient EL curves, all of the cables that connected the perovskite LEDs and oscilloscope needed to be as short as possible; the perovskite LEDs, the sample resistor and the oscilloscope were connected with a fast Bayonet Neill–Concelman connector to minimize the influence of the inductance of the circuit.

Transient photocurrent measurements of near-infrared perovskite photodetectors

The TPC data of the photodetector were measured using an oscilloscope with a 50 Ω resistor as the input impedance, and the device was excited by a pulsed laser (355 nm; pulse width, 10 ns) from a semiconductor laser diode with a neutral-density filter. When characterizing the TPC signal, all of the cables that connected the perovskite photodetectors and oscilloscope needed to be as short as possible; the perovskite photodetectors, the sample resistor and the oscilloscope were connected with a fast Bayonet Neill–Concelman connector to minimize the influence of inductance of the circuit.

Scanning electron microscopy and high-angle annular dark field scanning transmission electron microscopy measurements

Scanning electron microscopy measurements were carried out using a field-emission scanning electron microscope (Hitachi SU8010). The near-infrared perovskite films used in the measurements were deposited on glass/ITO/PEIE-modified ZnO. The cross-sectional samples were prepared by using a dual-beam focused-ion-beam system (Quanta 3D FEG). The device structures and perovskite layer were characterized by a Cs aberration-corrected Titan microscope at 200 kV (G2 80-200 ChemiSTEM). The cross-sectional high-angle annular dark field scanning transmission electron microscopy images and elemental mapping results were obtained using the aberration-corrected FEI Titan G2 80-200 ChemiSTEM instrument.

Photonic neural network training

We implement the photonic neural networks with PyTorch. All the networks are trained using the Adam optimizer. The complex amplitude of the light propagating through the waveguides is represented as complex-valued tensors. We designed the trainable tensors to represent the phase shift values, initially defined in real terms. For practical experimental demonstrations, these values were adjusted to fall within the 0–2π range. The network’s optimization leverages standard backpropagation techniques, enabling the automatic calculation of gradients for phase shift parameters. We incorporated a model of the nonlinear activation function, calibrated using parameters derived from empirical studies of saturable absorption. Our training approach for the AMCD tasks excluded data augmentation techniques. However, for the nonlinear photonic simulation task, we implemented a reshuffling of both inputs and outputs, maintaining consistent order due to the limited dataset available. We managed the randomization aspect of training by setting specific seeds for PyTorch. For the first four training rounds (encompassing tasks like AMCD, edge detection, MNIST and CIFAR-10), we sequentially set PyTorch’s random seed from 0 to 3. For the nonlinear photonic simulation task, we consistently reset PyTorch’s random seed to 0 for each distinct t value. This approach ensured controlled variability and reproducibility across our experiments.