In-band full-duplex (IBFD) communication technology transmits and receives signals simultaneously in the same frequency bands, theoretically doubling the spectrum efficiency. However, the leakage from the transmitter to the receiver leads to severe self-interference (SI) that must be eliminated. Conventionally, the SI signals are canceled in the electrical domain using electronic circuits, but due to the electronic bottleneck, it is difficult to realize the SI cancellation (SIC) of large bandwidth signals, with poor tunability. Photonics-assisted SIC methods have been proposed to break the electronic bottleneck. Nevertheless, considering multipath SI signals introduced by wireless channels or even complex multiple-input multiple-output (MIMO) scenarios, the existing photonics-assisted analog SIC schemes employ multiple parallel photonic links and a large number of delay and amplitude tuning devices to construct the reference signals for multipath SI signals. This is complex and difficult to track the rapid change of multipath channel response in actual wireless systems. The digital domain method is combined with the photonics-assisted SIC scheme as an auxiliary means to reduce the complexity of constructing the complex multipath SI signals and meet the multipath SIC requirements. However, in the IBFD MIMO system, besides the multipath SI signal, the nonlinear distortion caused by the power amplifier and crosstalk among different transmitting links will bring channel model changes. Till now, the photonics-assisted SIC scheme simultaneously considering the inter-channel crosstalk, nonlinear distortion, and multipath effect has not been studied, which should be urgently studied.

In IBFD MIMO communication systems, the inter-channel crosstalk, nonlinear distortion, and multipath effect collectively lead to exceptionally complex SI signals. To eliminate the complex SI signals in large bandwidth application scenarios, we propose a least square (LS) algorithm-assisted scheme for the cancellation of MIMO nonlinear SI in the optical domain and combine subsequent digital domain SIC. A continuous-wave light wave is modulated in a dual-drive Mach-Zehnder modulator (DD-MZM) by the received signal and the digitally constructed reference signal. The complex SI in the received signal can be suppressed after the optical signal from the DD-MZM is beaten in a photodetector (PD). To construct the analog reference signal, this method models the MIMO multipath SI signals in the presence of inter-channel crosstalk and nonlinear distortion. The model parameters are estimated by the LS algorithm, and then the analog reference signal for the analog optical domain SIC is constructed via the obtained model. Additionally, we reduce the order of the LS algorithm and improve the reference construction speed by setting a threshold and ignoring the components with low power in the SI signal while ensuring the analog SIC depth. Based on a two-step SIC of digital-assisted analog optical domain SIC and digital domain SIC, the complex SI signals in the IBFD MIMO communication systems can be well eliminated.

An IBFD MIMO scenario with two transmitting antennas is assumed in the experiment. The SI signal has a center frequency of 1 GHz, a baud rate of 0.5 Gbaud, and a signal duration of 3.8 μs. The dominant third-order nonlinear distortion is only considered and the SI signal from each antenna has seven multipath components. The crosstalk coefficient is first set to 0.1. After estimation by the LS algorithm, the tap coefficients of the filter are obtained and then normalized. Based on the normalized tap coefficients, the running time of the LS algorithm is tested. When the threshold of the normalized tap coefficients increases from 0 to 0.2, the running time of the algorithm in MATLAB is significantly reduced from around 0.25 s to 0.07 s. With the increasing threshold, the construction of complex SI signals using the LS algorithm will not be accurate enough, and the cancellation depth of the analog optical domain SIC will decrease from around 27 dB to around 15 dB. However, after further digital domain SIC, the overall SIC depth is around 35 dB, which is similar to that when the threshold is low and the digital domain SIC is also employed. When the crosstalk coefficient is set to 0.3, increasing the threshold of the normalized tap coefficients can also greatly reduce the running time of the algorithm in MATLAB. Due to the large inter-channel crosstalk in this case, which indicates relatively large multipath SI signal power, the effect of analog optical domain SIC does not decrease significantly during increasing the threshold from 0 to 0.2, and the SIC depth of optical domain analog SIC can be maintained at about 28 dB. The experimental results show that a reasonable setting of the order of LS algorithm adopted for analog optical domain SIC can reduce the order of parameter estimation and computational complexity, and improve the construction speed of analog reference signals.

We propose and experimentally demonstrate a low-complexity digital-assisted nonlinear analog optical domain SIC method for the IBFD MIMO communication systems. By utilizing this method, the complex SI signal in the IBFD MIMO communication systems can be well constructed, which can be leveraged for the analog optical domain SIC. Additionally, when the LS algorithm is adopted to construct the analog reference signal, the low-power components in the SI signal are ignored by setting a reasonable threshold to reduce the order of parameter estimation and computational complexity of the LS algorithm and improve reference construction speed. The experimental results show that the proposed method can eliminate the MIMO multipath SI signals with inter-channel crosstalk and nonlinear distortion, and achieve an SIC depth of about 35 dB after analog and digital SIC when the SI signal carrier frequency and baud rate are 1 GHz and 0.5 Gbaud respectively. The proposed method provides a promising solution for the optical domain elimination of complex multipath SI signals in IBFD MIMO communication systems.