Aiming at the application research of polar code in optical communication system, this paper proposes the combination of polar coding modulation and optical fiber Module Division Multiplexing (MDM) system. We construct an MDM system based on bit interleaved polar coding modulation, and study the performance of Maximum Likelihood (ML) detection algorithm. The simulation results show that the addition of polar code greatly improves the system performance. Compared with the MDM system without coding, the MDM system with bit interleaved polar coding modulation improves the system performance by about 8 dB under the same Mode Dependent Loss (MDL) . The paper also shows the performance of ML detection algorithm and Minimum Mean Square Error (MMSE) detection algorithm in MDM system. The impact of different mode loss on system performance is also studied. Compared with MMSE detection algorithm, ML detection algorithm improves the bit error performance by about 3 dB.