Vibration of carrier platform during travel will lead to the presence of geometric distortion of the images, which are obtained by optical panoramic imaging system. An image correction algorithm based on inertial navigation information was proposed, in order to correct the image. First of all, through the coordinate conversion scheme based on inertial navigation information, we can get the corresponding relationship for coordinate points between the corrected image and the original image. Then, via the edge-directed interpolation method, we can determine the gray value of each pixel in the image after the correction. Experimental results show that the algorithm can effectively achieve the goal of image correction for optical panoramic imaging system, suppress the edge blur effect of images, and improve the effect of image restoration.