On the basis of the fractal encoding technology with a higher speed, a new H.264 P frame prediction method and a corresponding fast coding system were proposed to decrease the video compression coding time and bit rate . Firstly, the main ideas of intra-/inter-prediction in H.264 and their advantages and disadvantages in P frame encoding were analyzed. Then, the new video compression method based on the fractal video compression and its characteristics were discussed. Furthermore, by combining the advantages of new standard H.264 with fractal video compression method, the P frame in H.264 was predicted by proposed method. Finally, the fractal coefficients coding and residual coding generated from fractal prediction encoding were given. Experimental results show that the proposed coding system has reduced the bit rate and the encoding time to 65% and 19% respectively comparing to those of the H.264 reference software JM15.1, while the Peak Signal Noise Ratio(PSNR) can be ignored (average reduce 0.09 dB). These results can satisfy the requirements of variable motion video sequences and can improve the overall performance of the H.264 video coding system.