Terahertz inline digital holography with lower photon energy and higher penetrating power to nonmetallic and nonpolar substances has attracted increasing attention. However, the zero- order diffraction light and conjugate image will degrade the details of reconstructed image and cause the loss of image quality. The applications of phase retrieval algorithm in the visible and terahertz digital holography are compared and analyzed by simulation, and the imaging method that the number of hologram recording is reduced from two to one can be verified after studying the mean square error of reconstructed image changing with the recording distance and recording interval. The influence of noise on the simplified imaging is discussed, and the results show that when the hologram and background are both contaminated by noise, the phase cannot be restored to the predetermined value and with the increase of noise, the reconstructed image becomes more and more blurred until the image is covered completely by noise.