Passive harmonic mode-locking of erbium-doped fiber laser with graphene as a saturable absorber is proposed. Graphene saturable absorber is obtained via mechanical exfoliation method by peeling highly oriented pyrolytic graphite repeatedly. When the pump power is 135 mW, the laser operates at fundamental repetition rate of 1.646 MHz with pulse duration of 1.82 ps and 3 dB bandwidth of 1.7 nm. With the pump power and adjusting polarization in the cavity appropriately increasing, the laser operates at several harmonics of fundamental repetition rate and the highest repetition rate achieved is 77.36 MHz, which corresponds to the 47th harmonic. The output power, pulse duration and single pulse energy for different harmonics have been studied.