The sparse regularized image restoration method based on animage group adopts the adaptive structure group dictionary to replace the traditional learning dictionary based on the entireimage block.However, because some parameters in the algorithm have not been optimized, the complexity of the algorithm remains relatively high.Therefore, this study proposed a sparse regularization image restoration method based on an adaptive image group in terms of roughness.First, global and local image roughnesses were calculated.Then, the number of self-adaptive regularization iterations was calculated according to the global roughness, and the number of samples required for learning the dictionary was adjusted based on the local roughness.Finally, the adaptive parameters were applied to the process of sparse regularization image restoration based on an image group.The method proposed in this study was applied to a case involving image restoration of text removal for images with different degrees of smoothness. The experimental results show that the efficiency of image restoration can be greatly improved when a similar restoration effect is guaranteed, particularly in relatively smooth images, where the speed-up ratio can reach nearly 30 times.