The characteristics of Human Visual System (HVS) on noticeable brightness difference and correlation properties in natural color images are brought into the image enhancement algorithm. Based on the HSV color space, a novel Unsharp Mask (UM) image enhancement method guided by Optimum Noticeable Difference (OND), called OND-UM, is proposed for the V component to bring about the edge and detailed information. The key procedures include that the OND curve is obtained by the numerical fitting technique and an adaptive gain function is designed with local variance. Especially, for the low-contrast regions, the proposed Adaptive Stretching Function (ASF) improves the global contrast. And the S component is histogram-equalized and H component remains the same. At last, the color restoration is conducted in RGB space on statistic data of color correlation properties. Experimental results show good performance of the OND-UM to enhance the edge and reduce noise sensitivity, and the enhanced images after the color restoration look more natural and vivid and are suitable for the human vision.