Fig. 1. Schematic diagram of AI for photonics and photonics for AI (photonics for AI includes integrated photonic circuit and free-space optical neural network)

Fig. 2. Overview of key technologies in intelligent photonics

Fig. 3. Schematic diagram of different neural network structures^[6]. (a) Operating principle of neurons; (b) structure of deep neural network; (c) schematic diagram of on-chip integrated photonic chip based on WDM technology; (d) schematic diagram of diffractive optical neural network

Fig. 4. Integrated PPU^［49］. (a) Schematic diagram of fully integrated PPU; (b) optical image of commercial tunable DFB laser chip for microcomb generation; (c) top-view image of AlGaAsOI microring (radius is 144 μm); (d) optical micrograph of chip

Fig. 5. Operation for real number WDM-based convolution system^［49］. (a) Measured optical spectra of microcombs with 2-FSR (182 GHz) spacing; (b) power deviations of the microcomb; (c) normalized kernel weights compare the voltage applied to a loop; (d) normalized spectra of microcomb before (red line segment) and after GDC application of weight matrix [0,0.5; 0.5,1]

Fig. 6. Convolution for image processing [schematic of photonic convolution and edge extraction results with two 2×2 kernels (Roberts operator), input image (house) from USC-SIPI image database used for input image]^［49］

Fig. 7. Photonic convolutional layer in a CNN^［49］. (a) Architecture of the convolution neural network (CNN) with three convolution kernel matrices (W₁ = [0.351, -0.729; -1, -0.094], W₂ = [0.551, 0.891; 0.761,1] and W₃ = [0.512,1; 0.978, 0.953]) for handwriting digit recognition, corresponding feature maps of image digit 1 are also illustrated; (b) output RF signal after sampling for digit 1 (gray dashed line and green solid line are the calculated (ideal) and experimentally obtained waveform, respectively); (c) scatter plot for convolution accuracy measurement with a fixed kernel matrix (inset is a residual error distribution histogram showing a standard deviation of 0.018); (d) confusion matrices of digit classification

Fig. 8. Principle of optical convolution operation based on multimode interference^［50］

Fig. 9. Experimental results of optical convolution^［50］. (a) Input images; (b) feature images obtained with computer; (c) feature images obtained with OCPU

Fig. 10. MNIST handwritten digital image classification demonstration^［50］. (a) Network structure of the CNN, contains optical convolution layer and electrically fully connected layer; (b) confusion matrix of recognizing 10000 digits in the MNIST test database (abscissa indicates true labels and ordinate indicates recognition results); (c) variation in simulation accuracy, experimental accuracy, and experimental cross entropy loss during 350 epochs of training

Fig. 11. Structure of PCNC^［70］. (a) Schematic of PCNC matrix core; (b) principle of PCNC weight bank; (c) schematic of a single PCNC unit; (d) scanning electron microscope image of a PCNC unit

Fig. 12. Image convolution implemented by PCNC matrix core^［70］. (a) Experimental setup diagram; (b) matrix convolution results of several classical image processing kernels

Fig. 13. Image recognition and classification task based on convolutional neural network^［70］. (a) AFHQ dataset experiment with 3-channel PCNC matrix core; (b) MNIST dataset experiment with 6-channel PCNC matrix core

Fig. 14. Schematic and logic diagrams of silicon-based integrated DONN^［51］. (a) Slab waveguide and its cross-sectional schematic diagram; (b) schematic of a silicon-based integrated DONN; (c) logic diagram of silicon-based integrated DONN mathematically describes the physical calculation process of silicon-based integrated DONN

Fig. 15. Microscopic structure, experimental process, and test results of DONN-I1 and DONN-I3 chips^［51］. (a) Microscopic structure diagram of DONN-I1 chip; (b) microscopic structure diagram of DONN-I3 chip; (c) experimental system error compensation and testing process; (d) image of DONN chip after packaging; (e) experimental test result of sample in the test set before error compensation of DONN-I3 system; (f) experimental test result of sample in the test set after error compensation of DONN-I3 system

Fig. 16. Structure of DONN-M3 chip and experimental flow and test results^［51］. (a) Pictures after wiring and packaging; (b) micrograph of DONN-M3 chip and scanning electron microscope image close-up of diffractive unit array; (c) confusion matrix of DONN-M3 chip; (d) recognition results for handwritten digits of DONN-M3 chip

Fig. 17. Conceptual diagram of on-chip computing power network application based on silicon-based integrated DONN computing power units^［51］

Fig. 18. New architecture for optical computing that combines training and computation^［52］. (a) Optical neural network model for multimodal classification; (b) structure of concept verification chip

Fig. 19. Training of the TDONN chip^［52］. (a) Flow diagram of the in situ training process; (b) cost function versus training iterations in the image classification task; (c)–(e) normalized power distributions of on-chip diffractive units; (f) conceptual diagram of the drop-out algorithm; (g) difference in the number of iterations before and after deploying the drop-out algorithm at different levels

Fig. 20. Inference results of multimodal data^［52］. Inference probability distribution of (a)‒(d) visual information, (e)‒(h) audio information, and (i)‒(l) tactile information; confusion matrix of 100 test data for (m) visual information, (n) audio information, and (o) tactile information

Fig. 21. Schematic diagram and experimental setup of visible light D²NN^［54］. (a) Schematic diagram of visible light D²NN principle; (b) five-layer phase distribution; (c) experimental schematic; (d)(e) experimental setups

Fig. 22. Handwritten digit classification and recognition results in the visible light band^［54］. (a) Different handwritten digit targets; (b) simulation and experimental results; (c) comparison chart of energy percentage

Fig. 23. Schematic diagram of MW-D²NN and training model^[103]. (a) MW-D²NN with three layers; (b) training flowchart of multi wavelength neural network

Fig. 24. Recognition accuracy under different weight coefficients^[103]. (a) Recognition accuracy for first weighting parameters (α₁=0.3, α₂=0.3, and α₃=0.3); (b) recognition accuracy for second weighting parameters (α₁=0.8, α₂=0.1, and α₃=0.1); (c) recognition accuracy for third weighting parameters (α₁=0.396, α₂=0.332, and α₃=0.272); (d) recognition accuracy for fourth weighting parameters (α₁=0.396, α₂=0.272, and α₃=0.332)

Fig. 25. MW-D²NN simulation and experimental test results^[103]. (a1)‒(a3) Simulation results and (b1)‒(b3) experimental results with incident wavelengths of 635 nm, 532 nm, and 435 nm, respectively; (c1)‒(c3) simulation results and (d1)‒(d3) experimental results with incident wavelengths of 635 nm and 532 nm, 532 nm and 435 nm, and 635, 532, and 435 nm, respectively

Fig. 26. Schematic diagram of on-chip MDNN^［55］. (a) Optical layout of polarization-dependent object classification for the MDNN; (b) schematic of a single TiO₂ meta-unit with a fixed height H, while tunable structure dimensions D_x and D_y

Fig. 27. Microscopic structures of on-chip MDNN^［55］. (a) Top-view; (b) oblique-view; (c) false-color cross-sectional view

Fig. 28. Experimental demonstration of the on-chip MDNN^［55］. (a) Exploded schematic diagram of the MDNN integrated with a CMOS chip; (b) physical photograph of the on-chip MDNN (enlarged image is a optical micrograph of the fabricated MDNN built on a CMOS imaging sensor); (c) fabricated MDNN (1st and 4th columns), the output field intensity detected by the CMOS imaging sensor for these four-category classification MDNN in x- or y-polarization (2nd and 5th columns), and experimentally detected energy distribution (3rd and 6th columns); (d) experimental confusion matrices for MNIST and Fashion-MNIST classification, respectively, with 80 images randomly selected from the correct set of simulations, for counting in the four classes (i.e., 20 per class)

Fig. 29. Operation principle of PSNN for patter recognition^[135]. (a) Input patterns; (b) PSNN; (c) spatial-temporal target; (d) proposed time-multiplexed spike encoding; (e) target output response; (f) nonlinear computation mechanism of a photonic spiking neuron chip; (g) chip layout of integrated FP-SA chip

Fig. 30. Structure of the DFB-SA array chip^[56]. (a) Operation principle of photonic neuron-synaptic core; (b) micrograph picture of the fabricated DFB-SA array chip and (c) compact packaged module; (d) spike peak amplitude as a function of gain current

Fig. 31. Identification and cascading results. (a)‒(c) “X”“D”“U”; (d)‒(f) “N”“J”“U”

Fig. 32. Experimental results of matrix multiplication of four channel DFB-SA laser array^[56]

Fig. 33. Basic framework of the reservoir computing

Fig. 34. Implementation of deep PRC^[144]. (a) Structural diagram; (b) experimental setup

Fig. 35. Performance testing of PRC^[144]. Examples of signal sequences (a) at the transmitter and (b) at the receiver; (c) PRC measurement performance at different depths

Fig. 36. Implementation of a four-channel RC system based on a DFB array^[57]. (a) Schematic diagram of a four-channel RC system based on a DFB array ; (b) experimental setup of a four-channel RC system based on a DFB array ; (c) theoretical schematic diagram of a single-channel RC system based on a DFB; (d) four-channel DFB array used in experiment ; (e) spectrum of four-channel DFB array used in four-channel RC system based on DFB array; (f) PI curves of four-channel DFB array

Fig. 37. Experimental results^[57]. (a) SER of systems as a function of attenuation coefficient of injection power; (b) SER of systems as a function of attenuation coefficient of feedback power

Fig. 38. Implementation of photonic RC system based on VCSEL^[147]. (a) Chaotic system of a multi-delay mutual coupling VCSEL ring network structure for chaotic signal generation; (b) conceptual diagram of the RC; (c) photonic RC system based on VCSEL for chaos time series prediction

Fig. 39. Time domain prediction results^[147]. (a) 1-step-ahead task; (b) 2-step-ahead task; (c) 3-step-ahead task; (d) 4-step-ahead task

Fig. 40. Information processing pipeline in quantum neural networks

Fig. 41. General structure of quantum neural network ^[161]

Fig. 42. Working principle of the BB84 protocol ^[174]

Fig. 43. Optimization of quantum key distribution experimental parameters with fully connected neural networks^[176]

Fig. 44. Control feedback framework in quantum systems

Fig. 45. Accurate decoding of surface encoding information with fully connected feedforward neural networks ^[186]

Fig. 46. Reconfigurable diffractive optoelectronic processor^[58]. (a) DPU is a large-scale perceptron-like optoelectronic computing building block that can be programmed to construct different DNNs; (b) DPU is implemented using programmable optoelectronic devices, with core components including DMD, phase SLM, and CMOS sensors; (c)‒(e) three different types of neural network architectures were constructed, including the D²NN, D-NIN, and D-RNN

Fig. 47. All-analog photoelectronic chip ACCEL^［29］. (a) Workflow of ACCEL; (b) schematic of ACCEL with an OAC integrated directly in front of an EAC circuit for high-speed, low-energy processing of vision tasks; (c) confusion matrixes of ACCEL tested on the MNIST, Fashion-MNIST, and KMNIST datasets

Fig. 48. Large-scale diffractive-interference hybrid chiplet Taichi^［59］. (a) Distributed architecture forms a shallow-in-depth but broad-in-width network for Taichi; (b) schematic of Taichi layout and on-chip computing pipeline; (c) Taichi experimentally achieved on-chip thousand-category-level classification and high-fidelity artificial intelligence-generated content

Fig. 49. Schematic diagram of PhysenNet^[214]. (a) Schematic diagram of phase imaging based on model-driven deep learning; (b) predicted intensity y^ during iteration; (c) predicted phase x^ by DNN during iteration

Fig. 50. Schematic diagram of single-pixel imaging based on physics-driven fine-tuning^[218]. (a) Data-driven pre-training DNN Dθ and masks H; (b) single-pixel imaging system; (c) model-driven fine-tuning pre-trained DNN Dθ*

Fig. 51. Experimental reconstructed results^［218］. (a) Hadamard SPI with β=100%; (b) Hadamard SPI with β=6.25%; (c) deep convolutional autoencoder network; (d) TVAL3; (e) model-driven deep learning; (f) DGI with learned patterns H*; (g) pre-trained Dθ*; (h) pre-trained and fine-tuned Dθ**

Fig. 52. Generation and optical reconstruction process of 4K phase-only holograms by the 4K-DMDNet^［233］

Fig. 53. Computer-generated holographic display system^［233］. (a) Experimental setup for holographic display; (b) schematic diagram of the time multiplexing method for full-color display

Fig. 54. Reconstruction effect of different algorithms^［233］. (a) GS algorithm; (b) Holo-Encoder algorithm; (c) 4K-DMDNet algorithm; (d) comparison of running time and image quality of different algorithms

Fig. 55. Structures of super-resolution reconstruction networks for SIM. (a) Network architecture diagram of U-net-SIM^[240]; (b) Network architecture diagram of PINN^[242]

Fig. 56. ADMM-DRE-SIM implementation of resolution enhancement principle diagram^［250］

Fig. 57. Resolution enhancement of SIM images by ADMM-DRE-SIM^[250]. (a) Resolution enhancement of SIM image of tubulins; (b) resolution enhancement of SIM image of actins

Fig. 58. AI-enabled optical fiber sensing applications. (a) Applications of intelligent optical fiber sensing in geosciences and life sciences; (b) AI algorithm workflow for fiber sensing signals processing

Fig. 59. Schematic diagram of using semi supervised learning method to pick up earthquake arrival time^[271]. (a) Neural network architecture of PhaseNet-DAS; (b) two DAS arrays used to build training dataset; (c) residuals of differential arrival-times picked by PhaseNet-DAS for P waves and S waves

Fig. 60. Model transfer process based on adaptive decentralized AI (ADAI) technology^[273]. (a) Distribution characteristics of unlabeled data; (b) model transferring from the source domain to each target domain; (c) adaptive AI model selecting for each target domain; (d) multi-scenario application of ADAI technology in infrastructure security monitoring

Fig. 61. Signal processing of optical-fiber-sensor-assisted smartwatch^[265]. (a) Typical pulse wave signals with features defined; (b) estimation process of the blood pressure; (c) Pearson correlation coefficients for the SBP's seven features; (d) Pearson correlation coefficient matrix between the seven DBP features; (e) design of supervised neural network; (f) principle of back propagation neural network

Fig. 62. AI-integrated multiplexed optical fiber sensor for dynamic brain monitoring of multiple biomarkers in an ex vivo brain model^[264]. (a) Photographs of the ex vivo experimental setup; (b) algorithm flowchart of the AI model for biomarker level prediction; (c) pH value prediction using feature set 1; (d) temperature prediction using feature set 1; (e) DO prediction using feature set 2; (f) glucose concentration prediction using feature set 1; (g) multiplexed and dynamic monitoring for simulated TBI disease using an ex vivo lamb brain

Fig. 63. Modeling fiber femtosecond laser with AI technology^［290］. (a) AI model with a priori information feeding for intra-cavity evolution process inference; (b) workflow diagram

Fig. 64. Soliton formation comparison between SSFM and AI mode^［290］. (a) Full-field signal comparison at 100th roundtrip; (b) soliton formation process of SSFM (left) and AI model (right); (c) full-field signal comparison at 500th roundtrip

Fig. 65. Soliton molecule formation comparison between SSFM and AI model^［290］. (a) Full-field signal comparison at the 150th roundtrip; (b) soliton molecule formation process of SSFM (left) and AI model (right); (c) full-field signal comparison at the 500th roundtrip; (d) variations of inter-soliton separation (top curves) and inter-soliton relative phase (bottom curves) along roundtrips; (e) clockwise evolutionary trajectories of SSFM (left) and AI (right) in interaction plane