Image deraining is the process of reconstructing a high-definition background image by removing rain marks. The deep convolutional neural network is now the most widely used to eliminate rain streaks. The core of the convolution operation is parameters globally sharing, which remarkably reduces the amount of calculation and improves the generalization ability of the algorithm. However, this also makes the convolution operation unable to effectively consider the connections between local parts and the influence of the distant pixels on the operated region. This will result in an over-smoothing phenomenon in single image deraining. Inspired by the great success that graph network has achieved in recent years, we hope to improve the convolution method by combining the kernel idea of graph network. First, all pixels are treated as graph nodes, the similarity between neighboring pixels is estimated, and the threshold value determines whether or not an edge connection exists. After the graph structure construction is completed, the obtained adjacent matrix and the similarity matrix will be used during the convolution operation, the parameters of the convolution kernel are adjusted, and the connection between the pixels and the extraction of topological information are fully considered. The intensive comparison experiments of several states-of-the-arts on several benchmark datasets show the effectiveness of the proposed enhanced convolution, which can effectively promote the performance of various latest algorithms without increasing a lot of computing resources.