At present, the supervised person re-identification methods focus on the problem of single modality (visible image). However, in addition to visible images, there are a large number of infrared images which lack color and texture information in the 24-hour surveillance system. Therefore, the cross-modality pedestrian retrieval method can effectively improve the practicability of person re-identification technology. The current cross-modality person re-identification methods ignore the unique discriminant features from different modalities, which leads to the performance limitation. This paper proposes a cross-modality person re-identification method based on cross-modality identity mutual prediction learning and fine-grained feature learning. A modal specific identity classifier is designed to improve the discrimination and robustness of modal specific features. A cross learning mechanism is constructed to promote the network to transform the specific features of different modal into modal invariant features, so as to make effective use of the modal specific discriminant information. In addition, fine-grained feature learning further enhances the discrimination of network feature representation from both local and global aspects. Comparisons with the state-of-the-art methods on open datasets SYSU-MM01 and RegDB show the advantages of the proposed method.