Changes in various weather phenomena are accompanied by the movement of clouds. Continuous satellite cloud images obtained by meteorological satellites contain considerable spatio-temporal sequence information; that is, continuous satellite cloud images have significant time sequence characteristics, which can be used as basic information for cloud image prediction. Cloud image prediction is essentially a video prediction problem in which the spatio-temporal sequence characteristics of cloud images are analyzed and processed. To accurately predict the change in the position of clouds, by focusing on the unstable and nonlinear motion characteristics of clouds, a video prediction algorithm called SmartCrevNet for cloud image prediction based on the CrevNet video prediction model is proposed. In this algorithm, a spatiotemporal attention gated recurrent unit (STA-GRU), along with the lightweight attention module spatial group-wise enhance (SGE), is introduced into the original two-way autoencoder module of CrevNet to enhance the ability of feature extraction without increasing the amount of calculation. The algorithm was tested on the public dataset MovingMNIST and the FY-4A satellite cloud image dataset. The results show that, on the FY-4A satellite cloud image dataset and MovingMNIST dataset, the mean square error (MSE) of SmartCrevNet is, respectively, 7.3% and 6.1% lower than that of CrevNet, and the structural similarity (SSIM) is increased by 7.9% and 1.2%, respectively. The prediction effect is better than that of CrevNet and traditional video prediction algorithms.