Cross-domain face recognition (FR) has always been a research hotspot in the field of face recognition. It has high application value and development potential in real applications such as security and criminal investigation. The existing cross-domain face recognition methods usually establish the correlation between different domain faces in the image space or latent subspace, but ignore the intrinsic relation between the two, which easily leads to the loss of inter-modal correlation information. In order to solve this problem, in this paper, we propose a novel method, called Cross-Domain Representation Alignment (CDRA). CDRA algorithm explores the correlation between different domain face data in the face image space and latent space. First, in order to reduce information loss, the CDRA algorithm can learn the latent feature representation containing discriminant information by reconstructing the face in a single domain. Then, in image space, CDRA algorithm is used to cross domain from different domain latent features. In the latent space, CDRA directly aligns the latent feature representations of different domain by aligning the latent Gaussian distribution of different domain data, which promotes the feature representation to learn the cross domain information of different domain faces in different spatial dimensions and levels. Experimental results indicate the average face recognition accuracy rate of CDRA is 97.2% on Multi-Pie dataset, and 99.4% ± 0.2% on CASIA NIR-VIS 2.0 dataset. Simultaneously, the efficient cross-domain face synthesis is realized. The learned latent features of our CDRA method can obtain the essential cross-domain information in both image space and latent subspace for cross-domain FR task, which can effectively improve the cross-domain face recognition.