With the development of 5G and the emergence of multiple display terminals, image retargeting algorithms have received extensive attention. Most existing algorithms do not consider the aesthetic distribution of the image during retargeting, thus affecting the human visual aesthetic perception. In view of this situation, we propose an image aesthetic evaluation network based on multi-level attention fusion. The aesthetic information is obtained by extracting different fine-grained features and adaptively fusing them according to the attention mechanism. Then, the learned aesthetic information is combined with the saliency map, gradient map, and linear feature map of the image as the importance map to guide the multi-operation image retargeting algorithm. Experimental results show that the generated importance maps can well protect aesthetic information, and the obtained retargeting images has a better visual perception than the state-of-the-art methods.