Atomic force microscopy (AFM) is the main technique for measuring surfaces of sub-nanometer root-mean-square roughness, but its results always vary with the sampling conditions (sampling interval and sampling points) and measuring position. As a practical example of a supersmooth substrate random surface, an effective method based on accumulated power spectral analysis is proposed and evaluated for appropriate sampling term selecting. It is shown that this method can avoid the loss or redundancy of roughness information. A optimized measuring position selection strategy according to global and local topography analysis is proposed. It can reduce the uncertainties in surface roughness characterization due to characteristic differences of different local surfaces and the required number of measurements to obtain reliable results. The above researches provide an effective method for testing optical supersmooth substrate surface by atomic force microscopy.