Reconstructed image of digital holography has problems such as serious interference of speckle noise, low contrast and so on. Thus a method based on speckle reducing anisotropic diffusion (SRAD) model and nonsubsampled contourlet transform (NSCT) is proposed for improving reconstructed image quality of digital hologram. SRAD model is adopted to eliminate speckle noise of reconstructed image. After NSCT decomposition, a low-frequency sub-band and several high-frequency sub-bands are produced. Low-frequency sub-band coefficients are adjusted based on a nonlinear gain function and an image segmentation method. In high-frequency sub-bands, edges are enhanced using a NSCT modulus maximum edge detection method. A large number of experimental results show that, compared with nonlinear diffusion denoising methods and NSCT enhancement methods proposed recently, the proposed method can more effectively eliminate speckle noise and improve the contrast of reconstructed image. Furthermore, the edges are smooth and clear. As a result, the accuracy of recognition and measurement in digital holography can be improved.