The total variation image inpainting algorithm needs more iterations to remove the text and scratches. It is not ideal for the preservation of edge details, and has a staircase effect. The peak signal to noise ratio of the restored image is relatively low. Aiming at these defects, an improved algorithm is proposed. The logarithmic operation and additional directional gradient operator are added into the regular term of the original algorithm to avoid the staircase effect, reduce the number of iterations, and reduce the smoothing of image at edge details. Simulation results show that the improved algorithm reduces the number of iterations retains the details of the image, well and the restored image has good visual effect.