In recent years, with the development of computer technology, artificial intelligence has gradually penetrated many aspects of current human life. As the basic architecture of artificial intelligence, machine learning and neural networks have been given increasingly more attention to by researchers in recent years. At present, neural networks have been applied in matrix calculation, equation solving, data analysis, and many other fields, and have become a research field with great development potential in the 21st century. In conventional neural networks, linear functions are the primary mathematical tool. Nowadays, nonlinear activation functions (NAFs) can be employed to describe a large number of different systems, such as electric power systems, optical systems, economic systems, biological systems, computer networks, and communication systems. Nonlinear functions are a much more powerful mathematical tool than linear functions. Therefore, they are introduced into the neural networks to apply the neural networks to more nonlinear models.

Currently, the application of nonlinear functions in neural networks is mainly realized through the nonlinear activation layers. The nonlinear activation layers of neural networks (NNs) can alter the linear transformation relationship beteween multi-layer networks, and thus enabling the NNs to solve more complex and advanced learning with flexibility. In pursuit of faster processing speed and lower energy consumption, optical neural networks (ONNs) have caught much attention from researchers in recent years. The response time of photons in ONNs is often picoseconds, and the energy loss of optical systems is often lower than that of electronic systems. Thus, the design of ONNs features high throughput and low power consumption. Therefore, except for electronic systems, photonic systems have shown a broad development prospect in computing. Additionally, the intelligent photonics represented by ONNs has also emerged and become an important development direction of information processing in the future. As an indispensable module in ONNs, a series of optical NAF devices have emerged. The nonlinear phenomena caused by the interaction between strong light and medium provide a powerful theoretical basis for applying NAF in photonic network architecture, and the integrated photonic device is also a feasible experimental platform for realizing NAFs. The NAFs broaden the application range of ONNs and provide a potential way to construct the next generation of integrated photonic devices on chip, which has a very broad development prospect. In this review, we summarize the recent studies that introduce optical NAFs to the systems and discuss their physical mechanisms and application capabilities. Meanwhile, this review also summarizes and discusses the challenges and future trends for the development in the research on optical NAF devices in ONNs, and provides outlooks related to such devices.

Methods and principles for generating various NAFs under optic-electro-optic (O-E-O) and all optical regimes, which are the two major regimes reported so far, are summarized with more emphasis on the later regime. The applications for NAFs are also presented. NAF modulators under the O-E-O regime can be dissected into electro-absorption-modulators (EAMs) and electro-optic-modulators (EOMs), both of which have distinct merits and drawbacks. EOM utilizes a Mach-Zehnder interferometer (MZI) or the phase shifts in a micro-ring resonator (MRR) to modulate amplitude via interference. EAM can directly modulate the light amplitude without the need for interference and thus can be designed with a smaller footprint. However, the carrier-based EAM suffers from lower speed compared with the field-driven EOM. More specific comparison can be seen in the main content (Fig. 14). Besides the aforementioned categories, O-E-O modulation can also be achieved by doping the MRR to make its transmission sensitive to electrical currents and thus can engender different NAFs under different biasing conditions. NAF modulators under the all optical regime can be dissected into three major categories including customized materials, semiconductor optical amplifier (SOA), and MRR. Customized materials can be further divided into saturated absorption, reverse saturated absorption materials, electromagnetically induced transparency (EIT) materials, phase change materials (PCMs), and light matter interaction (LMI) material structure. Meanwhile, we provide the mathematical basis of the cross-phase and cross-gain modulation effects in SOA. Additionally, examples of how these nonlinear effects can be utilized to realize optical neural-network devices capable of simple operations are provided by chaining multiple semiconductor optical amplifiers together in certain configurations, such as optical "AND" logic gates and optical signal thresholders. For MRR, by utilizing the free carrier dispersion (FCD) effect or thermo-optical (TO) effect, NAFs with distinct responding times and threshold can be designed and optimized by incorporating materials like graphene or Ge and platforms like Si₃N₄. The Kerr effect in graphene can enhance the FCD in silicon, while Ge can be adopted to facilitate the TO process. In addition, the Si₃N₄ platform can be utilized to increase the processing speed by blocking FCD and turning to the Kerr effect. Performance parameters relevant to the threshold, responding time, and loss of these devices are also summarized and compared. The generated NAF can be leveraged to improve the performance in applications like pattern recognition and classification while adding reconfigurability to the NNs and facilitating real-time response NNs and efficient information processors. Finally, the prospects for NAF development, including reconfigurability, better performance, and developments in combination with quantum information processing are put forward.

NAFs in the optical regime have been realized with various schemes and enhanced the performance of ONNs. In summary, the performance of optical NAFs still needs improvement in terms of faster responding time or lower threshold and loss whether by incorporating new materials or by deliberately designing the SOA or MRR systems. As a result, they can better serve ONN to perform more accurate and complex tasks.