In this paper, image inpainting is applied to image compression and a novel scheme of compression based on edge information is proposed. In encoding, Edge extension image is only encoded so that required information for encoding is greatly reduced, and thus a high compression rate is achieved. In decoding, a harmonic inpainting model is applied to reconstruct the image. Finally, a lot of images including gray level and color images are simulated. The results show that with this scheme proposed, we can get a good quality of reconstructed image in the condition of less image information.