Multiple Input Multiple Output (MIMO) detection algorithm is used to compensate the Mode Dependent Loss (MDL) in Mode Division Multiplexing (MDM) optical fiber channel. The problem of MIMO detection algorithm is that the computational complexity is too high and the performance is degraded with a large number of modes. This paper proposes to apply Massive-MIMO detection algorithms to the fiber channel, The algorithms include Minimum Mean Square Error (MMSE), Conjugate Gradient (CG), Gauss Seidel (GS), and Alternating Direction Method of multipliers based Infinity-Norm (ADMIN), and Optimized Coordinate Descent based BOX constraint (OCDBOX) . The result shows that the OCDBOX has the best Bit Error Rate (BER) performance in MDL-impaired MDM optical fiber communication system while higher complexity. The ADMIN has the second best BER performance and lower computational complexity. Therefore, the ADMIN detection algorithm can represent a good candidate in the MDM system with MDL.