A novel image encryption method based on iterative amplitude-phase retrieval and nonlinear double random phase encoding is proposed. In this method, a fake image is first used to generate the cipher text with the help of two public keys in the nonlinear double random phase encoding scheme. Two private keys are generated in the encryption process, where the original image to be encoded and the cipher text are applied as two constraints in the fast iterative amplitude-phase retrieval algorithm. The decryption processes can be finished in linear doubled random-phase encoding system based on 4f system. This encryption method has the advantages of fast convergence speed and high security. The proposed image encryption method has a resistance against on the amplitude-phase retrieval-based attack. The theoretical analysis and simulation experiments both validate the feasibility and security of the proposed scheme.