The effect of modulation instability in the multi-photon nonlinear Compton scat-tering on wavelength-division multiplexing systems is studied using small signal analysis. The result shows that the effect of modulation instability on short waves is stronger than that of on long wave;it will cause a deviation in each channel's SNR and the deviation will be accumulated with communication fiber length.