Aiming at the phenomenon that the accuracy of convolutional neural network is easy to be saturated in gait recognition and the problem of low fitting efficiency of vision transformer (ViT) to gait data set, an idea to construct a symmetrical dual attention mechanism model was proposed to retain the time order of walking posture, and fit the gait image blocks with several independent feature subspaces. At the same time, the symmetrical architecture was adopted to enhance the role of attention module in fitting gait features, and the heterogeneous transfer learning was used to further improve the efficiency of feature fitting. The model was applied to CASIA C infrared human body gait database of Chinese Academy of Sciences for many simulation experiments, and the average recognition accuracy was 96.8%. The results show that the proposed model is superior to the traditional ViT model and CNN comparison model in stability, data fitting speed and recognition accuracy.