The decrease of the frequency response of the optical transfer function in the optical synthetic aperture imaging system will inevitably lead to image blur. Therefore, it is usually necessary to use the Wiener filtering or blind deconvolution algorithm to achieve image restoration, and clear and high-resolution images are obtained finally. A deep learning frame based on a U-shaped convolutional neural network is proposed. The data set is constructed by the MATLAB software to train the network. The image restoration effects of the trained U-shaped network and blind deconvolution algorithm are compared. The numerical simulation results show that the U-shaped network has strong recovery ability, generalization ability, and versatility in the image restoration based on the optical synthetic aperture imaging system under the condition of weak noise. It can realize fast blind restoration for images and has potential application prospects.