For the inherent misregistration between the two sets of spectral bands and its effect on the spectral purity of each pixel hyperspectral imagery, a sub-pixel image registration method was proposed based on the characteristic of the hyperspectral imagery. Firstly, band selection and principal component transformation were used to process the Visible Near Infrared (VNIR) and Short Wave Infrared (SWIR) spectral bands, and the first principal component was chosen as the input image. Then, the input image was evenly divided into the image block with a certain overlap rate. Furthermore, the phase correlation method was taken to estimate the sub-pixel motion and the phase correlation coefficient was used to remove a few mismatches in the correlation output and to generate an optical flow field. Finally, the spectrometer images were registered by the calculated optical flow filed. Experimental results show that the accuracy of registration is superior to 0.1 pixel, which meets the accuracy requirement of advanced image processing.