With the rapid advancement of emerging technologies such as artificial intelligence, cloud computing, 6G mobile communications, and data center interconnection, modern optical communication systems are facing unprecedented challenges in transmission capacity, speed, and distance. In particular, long-haul and unrepeatered transmission systems are critically constrained by the accumulation of Kerr nonlinear effects—especially self-phase modulation (SPM)—as transmission distance and launch power increase. This accumulation causes severe signal distortion and elevated bit error rates, significantly affecting system stability and reliability. Although digital backpropagation (DBP) and optical phase conjugation (OPC) have been proposed to mitigate such impairments, their practical application in unrepeatered links is limited due to excessive hardware complexity or strict requirements on link symmetry. To address these issues, a low-complexity, hardware-independent compensation scheme called Phase Conjugate Twin-Wave Optimization for Unrepeatered Transmission (PCTWO-UT) is proposed, tailored for long-distance fiber links without inline amplification.

The core idea of the PCTWO-UT scheme is to construct a pair of conjugated signals over orthogonal polarizations (x and y polarizations) at the transmitter. These signals propagate through the fiber and undergo opposite nonlinear evolutions. At the receiver, they are coherently summed to cancel first-order Kerr nonlinear distortions. Compared with traditional PCTW methods, which are sensitive to polarization variations, PCTWO-UT introduces mutually uncorrelated QPSK training sequences in both polarization paths. This design enables full-rank channel estimation under polarization rotation conditions and resolves the convergence issue of constant modulus algorithm (CMA) based blind equalizers, which may fail when the input signal matrix is rank-deficient.

Furthermore, in high-speed or high-dispersion transmission scenarios, pulse broadening breaks the dispersion and power symmetry required for effective nonlinear compensation. To address this problem, the scheme incorporates distributed Raman amplification. By configuring pump wavelengths and power levels, a symmetric power distribution along the fiber can be achieved. Combining with pre-dispersion compensation at the transmitter helps restore the real-valued nature of the nonlinear transfer function, thereby improving the compensation effectiveness of the PCTWO-UT scheme under high-dispersion conditions.

A 60 Gbit/s PCTWO-UT-QPSK unrepeatered transmission system was established using a co-simulation platform based on VPI TransmissionMaker and Matlab. Simulation results show that in a 250 km standard single-mode fiber (SSMF) link, the proposed scheme achieves a maximum signal-to-noise ratio (SNR) improvement of 10.8 dB compared to a conventional single-polarization QPSK system affected by SPM (Fig. 2). This SNR gain is equivalent to tolerating an additional 64 km of fiber-induced loss. Within the practical launch power range (≤10 dBm), the system consistently maintains over 80% nonlinear compensation efficiency, demonstrating excellent robustness. Moreover, the scheme provides stable SPM suppression across different transmission distances (Fig. 3).

When the baud rate increases, the compensation performance degrades due to increased waveform distortion and the loss of the required link symmetry. As shown in Fig. 4, the scheme’s effectiveness sharply declines beyond 13 Gbaud. To further investigate the impact of chromatic dispersion, a 10 Gbaud signal was simulated over fibers with varying dispersion coefficients. Results reveal that when the dispersion exceeds 12 ps·nm^-1·km^-1, the compensation ability drops significantly (Fig. 5), emphasizing the need for dispersion-aware link design. While dispersion-shifted fibers can address this issue theoretically, standard single-mode fibers are more prevalent in practice. Therefore, this work adopts distributed Raman amplification as a more practical optimization approach.

In this configuration, the receiver-end EDFA is replaced with a second-order distributed Raman amplifier. Pump wavelengths of 1450 nm and 1360 nm are deployed with powers of 225 mW and 425 mW, respectively, to form a nearly symmetric power distribution across a 150 km SSMF span (Fig. 6). This power symmetry, combined with dispersion pre-compensation, significantly improves the system’s ability to suppress nonlinear effects. The constellation diagrams show reduced phase noise and improved clustering compared to the traditional EDFA-only setup (Fig. 7). Quantitatively, the nonlinear compensation gain increases from 2.3 dB to 5.2 dB, and the compensation efficiency rises from 32% to 62% (Fig. 8, Table 1), validating the effectiveness of Raman-based optimization in high-speed, high-dispersion scenarios.

This work proposes a PCTWO-UT-based nonlinear compensation scheme for long-haul unrepeatered coherent optical transmission systems. Based on phase-conjugated dual-polarization signal generation and coherent summation at the receiver, the scheme effectively cancels first-order SPM distortions without requiring additional optical hardware. The introduction of uncorrelated training sequences improves equalizer convergence under polarization rotation, while distributed Raman amplification restores power symmetry and extends the scheme's applicability to high-baud-rate, high-dispersion systems. Simulation results confirm that the proposed method significantly improves nonlinear tolerance and transmission performance, offering a practical and scalable compensation solution for future large-capacity, long-distance optical networks.