A Graphic processing Unit(GPU) optimization programming method is presented to apply to the real-time restoration of motion blurred video images. The blocks and threads run on the GPU are optimally set based on the hardware structure of Compute Unified Device Architecture (CUDA), and a memory access method is introduced to implement automatic coalesced access. These are required to make sure the full utilization of the GPU’s hardware resource. According to the symmetry property of FFT spectra, the redundant information in the frequency spectrum is eliminated and the number of frequency data filtered by the image algorithm is decreased,by which the amount of GPU memory access for realizing the algorithm optimization is reduced and the computing efficiency is improved. The experiment indicates that the proposed GPU project can improve the computing performance by 10 times as compared with the conventional CPU project, and the design of half-spectrum filtering can reduce the above time consumption by 20%. The experimental results confirm the feasibility and the validity of proposed method.