Acta Optica Sinica, Volume. 45, Issue 14, 1420016(2025)
Research Progress of Channel Waveform Modeling in Optical Transmission System Based on Deep Learning (Invited)
The exponential growth of data traffic has propelled optical networks towards wideband, high rate, and large capacities. Accurate and rapid optical fiber transmission simulation systems are essential for optimizing optical network configurations, developing advanced digital signal processing (DSP) algorithms, and performing end-to-end (E2E) global optimization. The optical fiber channel model plays a crucial role in simulation systems, as it describes the propagation process of optical signals within the optical fiber. The propagation of optical signals in optical fibers is governed by the nonlinear Schr?dinger equation (NLSE). Except in some special cases, the NLSE lacks an analytical solution and must be solved through numerical simulations.
The Gaussian noise model (GN model) and the enhanced Gaussian noise model (EGN model) provide precise and fast optical fiber channel modeling, primarily focused on power-level simulations. However, they cannot offer detailed waveform information, limiting their application in the design and optimization of DPS algorithms, especially for nonlinear compensation. The split-step Fourier method (SSFM) offers waveform-level channel modeling but requires extensive iterative calculations, with computational complexity scaling to the fourth power of bandwidth, limiting its applicability in wideband systems. Deep learning (DL) technologies, with their remarkable nonlinear fitting capabilities and efficient parallel computing, have demonstrated comparable accuracy to SSFM in optical fiber channel waveform modeling, while reducing computational time by one to two orders of magnitude.
This paper reviews recent advances in DL-based optical fiber channel waveform modeling techniques and categorize them from three perspectives: long-haul transmission modes, DL model structures, and incorporation of physical information (Fig. 3). We also present the principles and performance metrics of these approaches (Table 1).
In terms of long-haul transmission modes, DL schemes are classified into overall and distributed schemes. Overall schemes utilize a single DL model to represent the entire long-haul optical fiber channel, offering lower computational complexity and simplified data collection. However, they face challenges in handling amplified spontaneous emission (ASE) noise and achieving effective convergence. In contrast, distributed schemes employ multiple cascaded DL models to achieve long-haul transmission, each representing a single fiber span. This approach reduces the complexity of the channel effects the models must fit, improves accuracy, and simplifies the handling of ASE noise between models. By adjusting the number of models, distributed schemes allow for flexible distance generalization. Therefore, distributed schemes outperform overall schemes in handling ASE noise, achieving distance generalization, and improving model accuracy, making them the preferred method for waveform modeling in multi-channel wavelength division multiplexing (WDM) systems.
Regarding DL model structures, schemes can be divided into neural networks and neural operators. Neural networks, such as conditional generative adversarial network (CGAN), bi-directional long short-term memory (BiLSTM), and multi-head attention, demonstrate strong nonlinear fitting capabilities. Among them, BiLSTM and multi-head attention, as temporal neural networks, exhibit superior accuracy when modeling time-dependent optical fiber channel characteristics due to their recurrent structure and self-attention mechanisms. Neural operators, an emerging DL method, map between infinite-dimensional function spaces rather than discrete vector spaces, offering stronger generalization abilities compared to traditional neural networks.
In terms of incorporation of physical information, DL schemes are categorized pure data-driven and physics-data hybrid-driven methods. Pure data-driven methods require no complex domain-specific knowledge and simpler data processing and training processes. However, they demand larger datasets and longer training times, and may struggle to maintain high accuracy in multi-channel, high-rate WDM systems. Physics-data hybrid-driven methods combine physical information with data-driven approaches. There are two main strategies to incorporate physical knowledge. First, the physical constraint of the optical fiber channel, described by the NLSE, can be incorporated into the loss function, enhancing model interpretability and reducing the need for extensive datasets. Second, hybrid models combining physical models and DL models can jointly perform channel modeling, leveraging the interpretability of physical models and the nonlinear fitting capabilities of DL models for improved results. The physics-data hybrid-driven approach shows significant potential for scaling to multi-channel, high-rate WDM systems.
Over several years of development, DL-based optical fiber channel waveform modeling has emerged as a powerful technology, offering high accuracy and low complexity. It addresses the limitations of the traditional SSFM, which is plagued by high computational complexity, and becomes a key technology for future optical fiber channel waveform modeling. This paper first reviews recent advances in DL-based channel waveform modeling techniques, detailing their principles and performance metrics. We also explore the challenges of applying DL schemes to multi-channel, high-rate systems from the perspective of the more complex linear and nonlinear effects, as well as generalization of various system parameters. Additionally, we discuss potential optimization strategies from the perspective of incorporating more physical prior information, optimizing the structure of DL models, and improving the generalization capability of DL models. With ongoing technological advancements, we believe the challenges faced by DL approaches will be progressively overcome, ultimately positioning them as the dominant technology for channel waveform modeling in next-generation optical network.
Get Citation
Copy Citation Text
Minghui Shi, Zekun Niu, Hang Yang, Kaiyan Jin, Xinyi Zhou, Zhongyuan Sun, Zhaoyan Zhang, Lilin Yi, Lilin Yi. Research Progress of Channel Waveform Modeling in Optical Transmission System Based on Deep Learning (Invited)[J]. Acta Optica Sinica, 2025, 45(14): 1420016
Category: Optics in Computing
Received: Apr. 15, 2025
Accepted: May. 27, 2025
Published Online: Jul. 16, 2025
The Author Email: Zekun Niu (zekunniu@sjtu.edu.cn), Lilin Yi (lilinyi@sjtu.edu.cn)