The polarization dependent gain (PDG) of fiber Raman amplifiers (FRAs) with backward multiple pumps is investigated systematically, by numerical method based on the nonlinear coupled equations describing the properties of fiber Raman amplifiers. The PDGs induced by all waves Raman interaction, by pump-signal Raman interaction, by pump-pump Raman interaction and by signal-signal Raman interaction are studied respectively. The results show that compared to the FRAs with single pump, the FRAs with multiple pumps can obtain not only flat gains but also uniform PDGs. This study can be used to estimate the PDGs of a FRA with backward pumps induced by pump-signal Raman interaction, pump-pump Raman interaction and signal-signal Raman interaction. These results are useful and important for reducing the PDG of a FRA.