A novel optical encryption system is proposed by combining the optical principle of interference and diffraction. In this system, the original image is hidden into three masks, including two phase only masks (POMs) and one amplitude only mask (AOM). The AOM is randomly generated by computer while the POMs are obtained analytically. For decryption, the diffraction field of the two POMs is first superposed by utilizing the beam splitters and then modulated by the amplitude mask. After that the wavefront propagates for a certain distance and the intensity of the complex field, which is exactly the original image, can be recorded by CCD camera. Compared with the existing interference-based method, the approach is simple and easy to be realized by optical elements. Moreover, the silhouette problem that exists in the method with two POMs can be solved in our method. At the same time, the proposal is also proved to be robust to partial key exposure attack. Simulation results are presented to verify the validity of the proposed approach.