In common process of infrared image mosaic, using the traditional random sample consensus (RANSAC) method often takes a long time. In order to shorten the time consuming of image mosaic, a method of infrared image mosaic based on scale- invariant feature transform (SIFT) and improved RANSAC is proposed. Firstly, feature points and descriptors are obtained by SIFT algorithm. Then the wrong corresponding feature points are deleted by improved RANSAC. The image fusion is implemented, using the transformational matrix. In the improved RANSAC, we set thresholds of 0.95 and 0.85. After some iterations, we choose to jump out of the loop, reselect or calculate a new iteration number. The new number must be smaller than the old one, so the time consuming is shortened. Comparing the infrared image mosaic results with those results based on traditional RANSAC, we can find that the time consuming of proposed method is shorter.