Non-destructive, non-contact phase-shifting digital holography technology has distinct advantages in identifying micro-optical components. As traditional phase-shifting digital holography technology requires fine control and cumbersome calibration of the phase shifter, furthermore, its optical path is susceptible to mechanical vibration interference, which reduces the quality of the holographically reproduced image. To solve the above problems, we propose a vortex phase-shifting digital holography for the micro-optical element surface measurement with the help of the special phase distribution of vortex light. The method utilizes a helical phase plate to modulate the vortex phase and introduce a high-precision phase shift. Based on the constructed vortex phase-shifting digital holographic microscopy experimental setup, the actual phase shifts between phase-shift interferograms were determined using the interferometric polarity method, the relationship between the rotation angle of the helical phase plate and the phase shift was calibrated, and the feasibility of the vortex phase shift was experimentally verified. Repeated measurement experiments were carried out on the micro-lens arrays, and the measurement results were compared with those of the ZYGO white light interferometer. The results indicate that a single micro-lens's average longitudinal vector height is 12.897 μm with an average relative error of 0.155%. The proposed method enables highly precise measurement of the surface topography of micro-optical elements. It offers the advantages of easy operation, high stability, and high accuracy.