Addressing the issues of high algorithm complexity, large model overhead, and poor restoration performance in current convolutional neural network-based image restoration algorithms under object motion blur scenarios, we propose a lightweight image restoration model, SCDNet, based on a simplified dual self-adaptive serial attention mechanism. To reduce model complexity, we introduce the SimpleGate module, which splits feature maps into two parts along the channel dimension and multiplies them to reduce the model overhead caused by non-linear activation functions. We efficiently capture superpixel-level global dependencies using the simplified dual self-adaptive serial attention mechanism and adaptively transmit them to pixels to enhance the algorithm's pixel representation capability. Finally, by combining MS-SSIM and L1 loss functions, we better preserve image contrast, color, and brightness information, thereby improving image restoration quality. Experimental results show that, compared to the Restormer algorithm, SCDNet achieves a 0.30 increase in PSNR and a 0.12 increase in SSIM on the GoPro dataset, while its model parameters are only 22.4% of Restormer's.