There are obvious interference fringes in the push-sweep scanning image of temporally and spatially modulation spatial heterodyne interference imaging spectrometer (TS-SHIS), which will cause the traditional image registration method to have a great influence on the calculation results of TS-SHIS push-sweep scanning image registration. In view of this, an adaptive fringe template construction method based on target interference data is proposed, which is used to eliminate the interference fringes in the TS-SHIS push-sweep scanning image, and the surface fitting plus gradient method is used to register the stripe-eliminated push-sweep scanning image. Simulation and experimental results show that the proposed method can effectively eliminate the interference fringes at the zero optical path difference in TS-SHIS push-sweep scanning images, and the influence of interference fringes on registration calculation is suppressed. The influence of fringe elimination on image registration results is less than 0.02 pixel.