In order to solve the problems of image structure distortion and visual blur caused by insufficient utilization of hierarchical features between residual blocks, an image super-resolution reconstruction algorithm combining attention and residual aggregation is proposed. The network obtains multi-scale features through the shallow feature aggregation module and inputs them to the residual aggregation network, and adopts the progressive fusion strategy to aggregate the features of each residual block from both local and global aspects, so as to make full use of the level features of residual block. To further enhance the feature representation, a dual attention mechanism is used to focus on the interdependencies between features from space and channel, respectively. The experimental results show that, compared with SRCNN, FSRCNN and other methods, the reconstructed images have clearer structure and richer details.