For infrared small target detection and tracking, it requires very high efficiency of both hardware and algorithm. Since the classic background prediction algorithm is a serial one, which is very time consuming. Considering that common GPUs(Graphic Processing Units) are big in size and too power consuming to be integrated into an infrared device, the implement background prediction algorithm was proposed with separable convolution template method on the embedded GPU platform, named NVIDIA Jetson TK1. Taking advantage of CUDA(Compute Unified Device Architecture) programming language to execute background prediction algorithm in parallel, an operable and high performance result on board was achieved, which gained a 15x speedup comparing to the serial way with a CPU.