A brief overview of the basic principles and main applications of ultrafast photoemission electron microscopy is presented in this paper. Then, the progress of our research on ultrafast photoemission electron microscopy, including near-field imaging, near-field spectroscopy, and time-resolved ultrafast dynamics, in the field of nanophotonics, particularly in the field of surface plasmonics, is highlighted. These studies not only help to deepen understanding of the fundamental properties of surface plasmons and the interaction among different modes but also benefit the design and development of surface plasmonic applications. Finally, further potential applications of this technique are prospected.