Interferometry is an important method to realize high-precision measurements. The key requirement for achieving sub-nanometer resolutions is the interpolation and subdivision of periodic interference signals for hundreds or even thousands of times, through which the problem of resolution effectiveness is introduced. Based on the spiral phase property of the Laguerre-Gauss beam, modified high-precision interferometry was conducted using conjugate vortex beams. The measured linear displacement was linearly converted to the rotation of uniformly arranged petal-like interferograms. Regarding signal processing, to improve the measurement reliability, the multiple of the subsequent signal subdivision is effectively reduced via the subdivision of the spatial angles in interferograms. A high-speed photoelectric detection circuit and low-speed image processing were combined and separately used to count the number of integer cycles and image subdivisions to measure the rotation angle. The experimental test system was constructed using conjugate vortex beams with a topological charge value of 4, and 1°rotation of the interferogram corresponded to a theoretically measured displacement of 0.88 nm. The results of the experiment, which employed a real-time signal acquisition and processing system based on LabVIEW and error analysis, demonstrates that the resolution is better than 0.5 nm under normal laboratory conditions.