Visible light communication (VLC), as a new wireless communication technology, uses light-emitting diodes (LEDs) and other visible light sources to emit light signals, which are difficult for the human eye to detect and exhibit rapid light and dark variations for information transmission. VLC addresses the shortage of spectrum resources in radio frequency (RF) communication and can also integrate with existing RF systems, allowing operation in environments prone to electromagnetic interference. However, VLC faces significant nonlinear issues, primarily caused by the LEDs. Optical orthogonal frequency division multiplexing (O-OFDM) technology can effectively mitigate intersymbol interference in VLC and enhance spectrum utilization. However, due to the peak-to-average power ratio (PAPR) characteristics of O-OFDM, LED nonlinear distortion becomes more pronounced. Therefore, suppressing LED nonlinear distortion is critical to improving the performance of the O-OFDM system and advancing VLC’s practical application. To address this, we propose a post-equalization scheme, combining the least mean square (LMS) algorithm with a cascade neighborhood rough set (NRS)-improved convolutional neural network (CNN), bidirectional long short-term memory (BiLSTM) network, and self-attention mechanism to suppress LED nonlinear distortion. This approach classifies and optimizes the constellation points at the receiving end, mitigating the LED nonlinear distortion, improving the bit error rate (BER) performance, and reducing the computational complexity of the CNN?BiLSTM?Self-Attention model. Simulation results show that the proposed algorithm effectively alleviates LED nonlinear distortion in VLC.

The binary information sequence is modulated by multi-order orthogonal amplitude modulation (M-QAM) at the transmitting end, then output as a bipolar real signal through mapping and inverse fast Fourier transform. The signal is then processed through limiting, the addition of a loop prefix, parallel-to-serial conversion, and digital-to-analog conversion, followed by DC bias to drive the LED light. The receiving photodetector (PD) receives the optical signal, converts it into an electrical signal, and undergoes a processing sequence opposite to that of the transmitting end. The frequency-domain signal from the fast Fourier transform is then input into the cascade equalizer module to balance the signal and suppress the LED nonlinear distortion. Finally, the demodulator restores the original binary information sequence. In the cascade equalization module, the first-level LMS equalization reduces the dispersion of constellation points, facilitating the second-level enhancement of the CNN?BiLSTM?Self-Attention algorithm to improve classification accuracy and further reduce computational complexity. The NRS’s powerful classification decisions and attribute reduction ability, as well as its advantages in processing continuous data, are used to divide the constellation training set data into spatial regions and formulate the corresponding classification strategy. Through classification decisions, the accuracy of data feature extraction in the CNN?BiLSTM?Self-Attention algorithm is enhanced, LED nonlinear distortion is effectively suppressed, and algorithm complexity is reduced.

Monte Carlo simulation is applied to verify the performance of the proposed CNN?BiLSTM?Self-Attention equalization algorithm. First, compared to the classification strategy of the benchmark equalization algorithm, the proposed NRS classification strategy shows higher classification accuracy under low signal-to-noise ratio (SNR) conditions. For example, under 4QAM in ACO-OFDM and DCO-OFDM systems with an SNR of 10 dB, accuracy increases by about six percent points compared to the benchmark strategy (Fig. 11). Second, by setting the indoor VLC simulation parameters and the channel link model (Table 1 and Fig. 12) as well as the model hyperparameters and training the CNN?BiLSTM?Self-Attention deep learning model (Tables 2 and 3), the CNN?BiLSTM?Self-Attention model improved by NRS achieves optimal mean square error (MSE) and BER. This happens when the model parameter combination is set to sequence number 2 (Fig. 14). In addition, the number of training rounds also reaches optimal performance, confirming that the 10% training set is appropriate (Figs. 16 and 17). Furthermore, when the PD is positioned at the center of the room [3 m, 3 m, 0.85 m], the improved LMS cascade CNN?BiLSTM?Self-Attention equalization algorithm shows better BER performance. Compared with the benchmark equalization algorithm, the proposed LMS cascade improved CNN?BiLSTM?Self-Attention algorithm significantly reduces the bit error rate. For example, in the ACO-OFDM system, an SNR gain of about 7 dB and 11 dB is achieved for 4QAM and 16QAM, respectively, at a BER of 10^-4 (Fig. 18). Similarly, in the DCO-OFDM system, an SNR gain of about 7 dB and 14 dB is achieved for 4QAM and 16QAM, respectively, at a BER of 10^-4 (Fig. 19). However, at the same location, the BER performance of the DCO-OFDM system is worse than that of the ACO-OFDM system due to the increased DC component, which exacerbates LED nonlinear distortion. Moreover, the proposed cascade equalization algorithm shows better BER performance even at the edge of the room [0.5 m, 0.5 m, 0.85 m]. For example, in the ACO-OFDM system, 4QAM and 16QAM achieve SNR gains of about 12 dB and 13 dB, respectively, at a BER of 10^-4 (Fig. 20). However, compared to the center position, the BER at the edge is worse due to the poor channel gain. In addition, the improved LMS cascade CNN?BiLSTM?Self-Attention equalization algorithm has lower computational complexity than the traditional version (Table 5). For example, in the ACO-OFDM system with 4QAM and 16QAM, the complexity of the traditional CNN?BiLSTM?Self-Attention algorithm is 9.77×10⁸ at an SNR of 15 dB, while the improved algorithm’s complexity is 6.21×10⁷ and 6.42×10⁷, respectively. This is 1/16 of the traditional algorithm’s complexity. Under the same conditions, the traditional CNN?BiLSTM?Self-Attention algorithm for the DCO-OFDM system at an SNR of 15 dB is 3.56×10⁹, while the improved algorithm’s complexity is 2.09×10⁸ and 2.30×10⁸, respectively (Fig. 21).

In this paper, we propose the LMS cascade NRS-improved CNN?BiLSTM?Self-Attention equalization algorithm. It innovatively applies NRS in artificial intelligence particle computing to develop a constellation point classification strategy and enhances the CNN?BiLSTM?Self-Attention deep learning model. The CNN?BiLSTM?Self-Attention deep learning model improves constellation classification accuracy and reduces computational complexity. Simulation results demonstrate that the proposed approach outperforms the benchmark equalization algorithm, effectively suppressing LED nonlinear distortion in the VLC system, improving the system’s BER performance, and reducing computational complexity. Future studies can further enhance the robustness and stability of the proposed algorithm by investigating different channel models and dynamic nonlinear models of LEDs, thus improving its performance in more complex scenarios.