Polarization-dependent loss (PDL) of mode division multiplexing (MDM) links has a direct influence on the effective transmission of dual-polarization (DP) signals for large-capacity communication. In this paper, we aim to identify the origin of PDL in the MDM systems and optimize the transmission performance of DP signals. The PDL characteristic of the fundamental MDM system with a few-mode polarization controller (FMPC) is theoretically analyzed and verified by experiments. It is shown that the PDL of MDM links arrives at the minimum when the spatial pattern of mode channels is independent of the input polarization angle. The experimental data have good consistency with the theoretical curve. At the same time, the origin of PDL for MDM systems is identified, that is, the mode dependency can be converted into the polarization dependency in the MDM links. The theoretical and experimental results in the paper can provide guidance for PDL optimization of DP signals in MDM transmission.