The constant modulus algorithm (CMA) is a popular algorithm for mode-division multiplexing systems to equalize and compensate for impairments such as mode coupling, differential mode group delay, and dispersion in the system to obtain the desired signal. In order to study the equalization performance of the CMA in the strong coupling mode-division multiplexing system, the power coupling theory was used to build a 6×6 mode-division multiplexing system model and use the CMA and modified constant modulus algorithm (MCMA) at the receiving end to equalize the system output signal and obtain the constellation diagrams, root mean square error (RMSE) values and bit error rate (BER). The results show that in terms of the constellation diagram, MCMA can reduce scatter points and make constellation points more compact; in terms of RMSE, the RMSE value of the signal after MCMA equalization is smaller than the RMSE obtained after CMA equalization, indicating that the data dispersion level after MCMA equalization is low; in terms of BER, when BER is 10-3, the optical signal-to-noise ratio required by MCMA is 1.0 dB lower than that of CMA, therefore, MCMA equalization outperforms CMA. The results of this study provide some references for the equalization algorithm in the strong coupling mode-division multiplexing system.