The principle of error vector magnitude (EVM) against modulator nonlinearity for vector modulation signal (VMS) transmission in radio-over-fiber (RoF) systems is theoretically and experimentally investigated. A highly linear modulation scheme is proposed and demonstrated using a single-drive dual parallel Mach–Zehnder modulator (MZM). This method improves EVM performance and enlarges the linear input dynamic range of the VMS transmission. An index of maximum allowable input power difference (MAIPD) that reflects the difference of upper input power limits between these two schemes is measured. An EVM limitation of 5% MAIPD has 5 dB. Both 16- and 64-QAM results indicate that the proposed scheme supports VMS transmission better than the MZM one.