The exponential growth of data center interconnection (DCI) traffic, driven by cloud computing, AI-driven analytics, and hyperscale applications, has exposed critical limitations in conventional intensity modulation/direct detection (IM/DD) systems. While IM/DD systems remain widely adopted for their low power consumption and simplicity, their capacity and reach are fundamentally constrained by fiber chromatic dispersion (CD) and bandwidth limitation, particularly in single-wavelength 400 Gbit/s-and-beyond scenarios. Coherent optical interconnections, with their superior spectral efficiency (SE) and digital CD compensation capability, have emerged as a promising solution for next-generation 800 Gbit/s‒1.6 Tbit/s DCI system. However, the high cost and power overhead of high-speed analog-to-digital converters (ADCs) and digital signal processing (DSP) chips—key enablers of coherent detection—remain significant adoption barriers. Furthermore, achieving ultra-high SE through increasing symbol rates exacerbates inter-symbol interference (ISI) in bandwidth-constrained channels, demanding advanced receiver-side equalization that further escalates system complexity. Existing methodes to mitigate ISI, such as feed-forward equalizers (FFEs), maximum likelihood sequence estimation (MLSE), or transmitter-side pre-filtering, often trade off noise resilience for bandwidth efficiency, limiting their practicality in cost-sensitive DCI environments. We address these intertwined challenges by proposing a novel symbol-domain multiplexing (SDM) framework that simultaneously reduces bandwidth demand, enhances noise tolerance, and simplifies transceiver complexity. By rethinking conventional modulation paradigms, we aim to break the “bandwidth vs. complexity” deadlock in high-capacity DCI systems, providing a scalable pathway to 800 Gbit/s‒1.6 Tbit/s deployments while alleviating ADC/DSP-related cost and power constraints.

To address these challenges, we propose a novel symbol multiplexing method based on joint coded modulation, specifically designed to enhance spectral efficiency and bandwidth/baud rate utilization in coherent optical interconnection systems. The proposed method builds a linear coded relationship between quadrature amplitude modulation (QAM) symbols using forward error correction (FEC) encoding. By applying a “many-to-one” symbol mapping strategy, every two 5 bit 32 QAM symbols are combined and mapped onto a single constellation point, effectively generating a symbol-multiplexed 16 QAM signal. This method increases the symbol capacity from the conventional 4 bit per 16 QAM symbol to 5 bit, resulting in a 25% improvement in spectral efficiency and bandwidth utilization per baud.

The symbol multiplexing process in this work goes beyond conventional modulation formats by integrating the benefits of joint coding and modulation. Traditional modulation methods treat coding and modulation as separate layers, but by tightly coupling them, symbol multiplexing allows for a more efficient representation of information. This strategy is particularly advantageous in short-reach coherent systems where channel bandwidth is constrained because of hardware and design limitations. As data rates increase, scaling up symbol rates leads to more severe ISI and higher demands on DSP and hardware. The proposed method mitigates these issues by reducing the required system bandwidth.

To support the proposed transmitter-side symbol multiplexing, a corresponding joint demodulation and decoding method is introduced at the receiver. This receiver architecture performs parallel iterative decoding, leveraging soft information exchange between the FEC decoder and the QAM demodulator. The iterative process refines the symbol likelihoods, effectively improving the extrinsic information transfer (EXIT) curves and boosting the overall decoding performance. This technique enhances the reliability of symbol decisions in the presence of channel noise and ISI, leading to better bit error rate (BER) performance at lower optical signal-to-noise ratios (OSNRs).

The theoretical performance of the proposed symbol multiplexing method is analyzed under bandwidth-constrained conditions. Simulation and analysis demonstrate that this method provides notable gains in noise tolerance compared to conventional modulation formats. Specifically, by enabling more bits to be transmitted per symbol while maintaining manageable constellation sizes, the method improves trade-offs between complexity, power, and performance.

To validate the feasibility and performance of the proposed method, a 400 Gbit/s short-reach coherent optical interconnection system is implemented and experimentally evaluated. In this setup, the symbol-multiplexed 16 QAM signal is transmitted over a limited-bandwidth link, and the joint demodulation-decoding algorithm is applied at the receiver. Experimental results show that, compared to a conventional 16 QAM signal with the same effective data rate, the symbol-multiplexed 16 QAM requires only 0.8× system bandwidth to achieve effective data transmission. Moreover, it delivers a 3.97 dB OSNR gain, confirming the method’s robustness and spectral efficiency advantages.

The outcomes of this work indicate that symbol multiplexing based on joint coded modulation holds strong potential as an enabling technology for next-generation high-speed optical interconnections, especially in environments such as cloud data centers where efficient bandwidth usage and low power consumption are critical. By enhancing bandwidth utilization and providing better resilience to noise and ISI, this method can significantly contribute to the scalability and energy efficiency of future optical interconnection systems.