Remote sensing image registration is a research of significance in the field of image processing. Large rotation difference between multimodal images seriously affects the improvement of the final quality of image matching. To solve this problema rotation difference correction method based on double branch neural network is proposed. Firstlywith the calculation of eigenvalue and orientation of phase congruency informationthe rotation feature vector of the image is extractednamed RVPC. After thata double branch neural network R-FCN is constructed and fully trained to predict the rotation difference angle between images. Furthermoretwo RVPC vectors are input into the network and we get an output prediction vector and then calculate the prediction angle. Finallythe image is affine corrected based on the angle. On public data set SEN1-2the training accuracy of network R-FCN reaches 98.17%.