The commercialization of the fifth-generation (5G) mobile communication industry has escalated the demand for efficient and cost-effective application technologies. The faster-than-Nyquist (FTN) transmission technology in coherent optical wireless communication systems has caught significant attention due to its high signal transmission rate and channel capacity enhancement capabilities. However, the receiving terminal of the coherent optical system exhibits a high level of complexity, with performance significantly influenced by turbulence-induced phase noise and inter-symbol interference (ISI) introduced by FTN. Therefore, it is imperative to conduct studies on low-complexity optimization algorithms to address these challenges. In this regard, a blind equalization scheme for FTN quadrature phase shift keying (QPSK) coherent optical wireless communication signals under exponential Weibull channels has been devised. This scheme optimizes the signal processing algorithm at the receiving end to minimize complexity costs. A novel algorithm named TS-DFE-CMA has been proposed to integrate decision-feedback equalization (DFE) into a constant modulus algorithm (CMA), thereby enhancing system resilience against various disturbances. Additionally, employing a two-step (TS) approach enhances step coefficient selection and transformation methods to control computational complexity, with improved system performance. Simulation results demonstrate that the proposed algorithm mitigates signal distortion caused by FTN ISI and turbulence effects.

The transmitted signal undergoes QPSK mapping, FTN shaping filtering, and digital-to-analogue conversion at the transmitter. It is then split into two orthogonal signals of I and Q, which are transmitted by an optical antenna into a turbulent channel following an exponential Weibull distribution model. The received optical signal is subjected to zero-phase coherent detection using a 90° mixer, followed by analogue-to-digital conversion. Subsequently, signal processing techniques such as equalization and phase recovery are applied to obtain the demodulated decision output signal. In the signal processing module, the CMA algorithm is employed to recover the amplitude information of the signal. To address the phase insensitivity drawback of the CMA algorithm, we utilize a cascaded Viterbi-Viterbi phase estimation (VVPE) device for compensating phase noise. Improvements have been made to this algorithm, including adopting TS step-size optimization and selecting an initial step-size coefficient corresponding to minimum error vector magnitude (EVM) for achieving rapid convergence. Once the EVM curve stabilizes, a smaller step-size coefficient is adopted to enhance convergence accuracy towards reaching the optimal solution of function and reduce steady-state value during convergence. Furthermore, to further compensate for strong nonlinear interference and signal distortion caused by turbulent channels, we introduce the decision feedback equalizer structure into the objective function update process of the constant modulus equalization algorithm replacing a single filter with two filters, with one for feedforward and another for feedback. The output from the feedback filter predicts and eliminates interference introduced by preceding symbols.

The proposed algorithm is evaluated by simulation and compared with three algorithms of DFE-CMA, TS-CMA-VVPE, and CMA-VVPE. The results indicate that the proposed algorithm outperforms the other three algorithms in terms of bit error rate (BER) performance in weak and moderate turbulence conditions (Fig. 8). According to the EVM curves for the four different algorithms, EVM values are better at weak turbulence than those at medium turbulence. Furthermore, the proposed TS-DFE-CMA algorithm yields optimal EVM performance in both weak and medium turbulence conditions (Fig. 9). In comparison to the TS-CMA-VVPE algorithm, the proposed algorithm improves BER performance by 18.37% and EVM performance by 4.74% under weak turbulence (Table 2). Additionally, it is demonstrated that the proposed algorithm performs better than the TS-CMA-VVPE algorithm in mitigating FTN ISI effects. However, after reducing the FTN acceleration parameter to 0.7, it is found that BER fails to meet the threshold requirement for forward error correction (Fig. 10). Meanwhile, at FTN acceleration factors of 1.0, 0.9, and 0.8 respectively (Table 3), compared to the TS-CMA-VVPE algorithm, the proposed algorithm improves BER performance by 15.62%, 16.071%, and 9.07%, and EVM performance by 4.72%, 6.49%, and 3.81%. Additionally, the TS-DFE-CMA algorithm exhibits a more favorable convergence of signal constellation diagram (Fig. 11). Simulation results confirm that algorithms incorporating decision feedback processes perform well in mitigating turbulent effects and FTN ISI. Furthermore, the proposed algorithm demonstrates the fastest convergence speed with optimal equalizer performance (Fig. 12). Furthermore, the complexity of the TS-optimized algorithm is effectively controlled (Table 4).

The proposed algorithm TS-DFE-CMA enhances the convergence speed and error performance of the equalizer by employing a TS step optimization method. This is crucial for handling the demand in FTN-QPSK coherent optical wireless communication systems for signal phase noise recovery and ISI cancellation in exponential Weibull channels.