Bundle adjustment is the key step in real-time mosaicking of remote-sensing imaging. It takes too much calculation time and decides the accuracy of the mosaicking system. Aiming at bundle adjustment algorithm in real-time mosaicking system of frame-sweep infrared imaging, a practical policy of bundle adjustment was proposed. The bundle adjustment between images of two adjacent sweep lines was used to register the lastest sweep line to the mosaicked image, which laid the foundation of real-time updating of mosaic image. The GPGPU platform was used to implement and accelerate, which improved the real-time performance of mosaicking system and also guaranteed the accuracy. By implementation of real-time bundle adjustment of frame-sweep images, the precision and real-time performance of the bundle adjustment algorithm were validated.