High-speed PAM4 direct-detection optical transmission systems are highly susceptible to multipath interference (MPI), which induces elevated bit error rates (BERs) even under normal link power conditions. Conventional impairment compensation algorithms, such as feed-forward equalization (FFE), exhibit limited capability in mitigating MPI. Consequently, MPI has emerged as one of the primary limiting factors for the performance of high-speed PAM4-based intensity modulation and direct detection (IMDD) systems. In the time domain, MPI causes stochastic signal fluctuations, while in the frequency domain, it manifests as low-frequency noise. To address these challenges, the exploration of novel MPI suppression scheme is essential for advancing the reliability and efficiency of next-generation optical communication systems.

MPI induces stochastic temporal fluctuations in optical signals, with the fluctuation amplitude escalating proportionally to signal intensity levels. Building on the characteristic of MPI, we propose an MPI suppression scheme based on joint probabilistic shaping and peak-mean compression (PS-PMC). At the transmitter, probabilistic shaping is employed to reduce the occurrence probability of high-level symbols, thereby decreasing the symbol error rate. At the receiver, from a time-domain perspective, the signal is compressed based on the peak and mean values of each data frame, effectively mitigating signal fluctuations induced by MPI.

Under laser linewidths of 0.1 MHz, 1 MHz, and 10 MHz, the proposed PS-PMC algorithm improves the MPI tolerance by 6.1 dB, 4.9 dB, and 3.7 dB, respectively. Compared to the baseline A1 algorithm, these enhancements correspond to additional gains of 2.2 dB, 2.0 dB, and 1.6 dB under the same linewidth conditions. Therefore, the PS-PMC algorithm significantly enhances the system’s tolerance to MPI. The receiver-side PMC algorithm achieves performance comparable to the A1 algorithm. However, as the laser linewidth increases, the MPI suppression effectiveness of both algorithms diminishes. This occurs because larger laser linewidths increase the frequency of MPI-induced temporal signal fluctuations, which reduces the efficacy of the PMC algorithm that relies on per-frame peak and mean processing. At the transmitter side, probabilistic shaping performs worse than the receiver-side PMC algorithm under small laser linewidth conditions. However, its performance remains more stable across varying laser linewidths, and it even outperforms the PMC algorithm when the laser linewidth is large. This is because probabilistic shaping directly adjusts the probability distribution of PAM4 symbol levels at the transmitter, reducing the proportion of high-level symbols. Consequently, it decreases the number of high-level symbol misinterpretations at the receiver, making it less affected by laser linewidth variations.

MPI has a significant impact on the performance of high-speed PAM4 direct detection optical transmission systems. We propose an MPI suppression scheme based on joint probabilistic shaping and peak-mean compression (PS-PMC). At the transmitter, probabilistic shaping is employed to reduce the occurrence probability of high-level symbols, thereby decreasing the symbol error rate. At the receiver, from a time-domain perspective, the signal is compressed based on the peak and mean values of each data frame, effectively mitigating signal fluctuations induced by MPI. Simulation results demonstrate that under different laser linewidths and signal-to-interference ratio (SIR) conditions, the proposed PS-PMC scheme significantly enhances the system’s MPI tolerance. For laser linewidths of 0.1 MHz, 1 MHz and 10 MHz, the proposed scheme improves the MPI tolerance of a 25 Gbaud PAM4 signal transmitted over 15 km by 6.1 dB, 4.9 dB and 3.7 dB, respectively.