Fig. 1. Common nonlinear activation functions. (a) Sigmoid; (b) ReLU; (c) Elu; (d) leaky ReLU; (e) Softplus; (f) Radial Basis

Fig. 2. Nonlinear activation function of O-E-O system. (a) Absorptivity of different systems varied with voltage^[9]; (b) voltage nonlinear activation function after the photodiode is connected^[10]; (c) schematic of implementing a nonlinear activation function scheme in EOM system and the nonlinear activation functions that can be formed^[11]; (d) MRR resonance peak varied with I_h, I_b, _{and photocurrent under different permutations of illumination^[12]; (e) schematic of MRR^[12]; (f) nonlinear line type under different bias conditions^[12]}

Fig. 3. Saturated absorption and reverse saturated absorption materials. (a) Schematic of layered structure of SESAM^[15]; (b) nonlinear activation function of SESAM and SOA, the nonlinear characteristics of SESAM are mainly reflected at low input power, while the nonlinear characteristics of SOA are mainly reflected at high input power^[15]; (c) schematic of all-optical reservoir system^[15]; (d) schematic of dynamic reservoir system, the duration of each input signal is T=Nθ^[15]; (e) schematic of C₆₀ energy level structure^[20]; (f) relationship between light intensity density and transmittance of high concentration C₆₀ as a desaturated absorber in PVA host film^[21]

Fig. 4. Nonlinear activation function caused by EIT. (a) Energy level arrangement of EIT material without coupling field^[26]; (b) energy level arrangement of EIT material with coupling field^[26]; (c) experimentally measured EIT phenomenon of Sr vapor^[27]; (d) EIT component construction^[6]

Fig. 5. Nonlinear activation function caused by LMI materials. (a) Schematic of LMI material structure^[25]; (b) when the input power density of nanorod and quantum dot systems changes, the extinction cross-section (left y-axis) and imaginary part of the refractive index (right y-axis) of the system also change^[25]; (c) nonlinear function of transmittance versus input power density when quantum components are placed at different locations on the waveguide^[15]; (d) schematic of end-selectivity of surfactants to gold nanomaterials; (e) microscopic image of nanorods and nanospheres dimer^[28]

Fig. 6. Nonlinear activation function caused by PCM. (a) Nonlinear conversion before and after optical synapses is achieved through PCM on the waveguide^[29]; (b) optical nonlinear activation function implemented by PCM^[5]; (c) neural network achieved by nonlinear processing of multiple sets of input pulses utilizing PCM^[5]

Fig. 7. SOA nonlinear activation function. (a) SOA-based MZI implements a logic AND gate^[32]; (b)(c) nonlinear relationship among SOA gain, input optical power, and bias current^［35］; (d) Sigmoid trigger function neural network combined with XPM and XGM^［36］; (e) on-chip SOA array neural network^[33]

Fig. 8. FCD-induced nonlinear activation function. (a)(b) Schematic and experimental setup of MZC device coupling with MZI with MRR^[8]; (c) four line types that can be achieved by the scheme; (d) diagram of the experimental apparatus^[38]; (e) system-generated Sigmoid function; (f)(g) schematic after adding graphene and achievable line patterns^[39]

Fig. 9. All-optical nonlinear activation functions of Ge/Si composites^［40-41］. (a) Schematic and experimental samples of Ge/Si composite runway MRR; (b) schematic for achieving a nonlinear response; (c) schematic of different wavelengths achieving different line types; (d) schematic of MRR and its cross-sectional view; (e) schematic of the transmission spectrum and the corresponding nonlinear activation function at the through and drop ports as the input optical power increases; (f) nonlinear activation functions achieved in the Si₃N₄ platform

Fig. 10. All-optical neural networks used to solve classification problems. (a) A ring resonator implementing a nonlinear activation function^［5］; (b) apply the activation function to implement the function of pattern recognition^[5]; (c) a neural network architecture that implements classification functions, demonstrating linear and nonlinear operations in the network^［6］; (d) nonlinear activation function of EIT cavity output^[6]

Fig. 11. Electro-optical neural networks used to solve classification problems^[10]. (a) Architecture of electro-optical neural networks; (b) free carrier-based absorption modulators and absorption rates in Si and ITO

Fig. 12. Reconfigurable neural network architecture^[45]. (a) Integrated optical neural networks applying reconfigurable activation functions; (b) results of training in the Iris dataset using different activation functions

Fig. 13. Classification function based on tunable nonlinear activation function. (a) Schematic of neural network to implement image classification^[2]; (b) comparison of classification accuracy of optical neural network with and without activation function^[11]

Fig. 14. Application of activation function in real-time response neural network and efficient information processor. (a) Evaluation results of key indicators of electro-optical modulators and absorption modulators in different material systems^[45]; (b) schematic and training results of nonlinear activation function of optical sensors exhibiting nanophotonic structures that induce transparency and reverse saturated absorption^[46]