Deep neural networks (DNNs) have revolutionized traditional approaches across numerous scientific and technological domains, demonstrating exceptional performance in computer vision, natural language processing, speech recognition, and recommendation systems. While these networks rely on sophisticated models with extensive parameters, the underlying computing hardware plays a crucial yet often overlooked role. Each advancement in DNNs correlates directly with hardware capability improvements. As contemporary computing chips approach Moore’s law limitations, the computational power requirements continue to escalate. Consequently, both academic and industrial sectors are investigating alternative physical computing platforms, including in-memory computing. Optical computing emerges as a promising solution, harnessing light’s inherent multidimensional properties and light-matter interactions to develop optical or optoelectronic information processing systems. With distinct advantages including low latency, high parallelism, low power consumption, and large bandwidth, optical computing distinguishes itself in the development of “non-Von Neumann” integrated storage-computation platforms, achieving notable progress in recent years.

Significant advancements have emerged in accelerating neural network computations through optical systems, specifically optical neural networks (ONNs). Researchers have developed programmable photonic chips implementing linear matrix-vector multiplication via arrays of Mach-Zehnder interferometers (MZIs), micro-ring resonators (MRRs), and phase change materials (PCMs). Furthermore, they have investigated novel optical computing platforms including multi-layer diffractive neural networks, scattering media, and multimode fibers. These optical computing systems utilize spatial, temporal, and frequency modes in optics, or parallel combinations thereof, consistently advancing computational performance in optical neural networks. Present research in optical computing emphasizes hardware development, with hardware innovations frequently garnering substantial attention. The integration of software algorithms with optical computing hardware’s physical characteristics has received relatively less focus. However, algorithmic progress has substantially enhanced optical neural networks, demonstrating that effective hardware-software synergy yields significant research achievements. This paper provides a systematic review of the relationship between optical computing hardware’s physical characteristics and software algorithms (Fig. 1). The analysis examines how algorithmic advances enhance optical computing hardware performance (algorithm-enhanced ONN), particularly focusing on optimization techniques improving hardware processing capacity and efficiency (Figs. 2?6). Additionally, it explores how optical computing hardware’s unique physical properties can integrate with and enhance algorithms, termed hardware-inspired optical neural networks (hardware-inspired ONN) (Figs. 7?9). The paper concludes with perspectives on optical neural network development trends.

Despite significant progress and achievements in ONNs, the field remains relatively nascent compared to established electronic neural networks. This review examines the integration of hardware physics and algorithms in ONN development. The analysis reveals that collaborative design between hardware physics and software algorithms is fundamental for maximizing optical computing potential in machine learning applications. The 2024 Nobel Prize in Physics recognition of the Hopfield model and Boltzmann machines emphasizes the significance of physics-machine learning bidirectional interactions. In optical computing, algorithmic advances have enhanced simulated optical computing system stability, while optical physical phenomena have inspired novel model architectures extending beyond traditional neural network frameworks. The optical nonlinear Schr?dinger equation, for instance, has been adapted into a trainable model for physics-informed neural networks, generating several innovative architectural approaches. Through examining hardware-algorithm synergies, this review aims to stimulate mutual engagement between optics and algorithm development researchers, advancing the interdisciplinary field of optical neural networks. Additionally, it seeks to establish a sustainable development trajectory for optical computing through dynamic optical physics and machine learning interactions.