In view of speckle noise in medical ultrasound image, a denosing method based on adaptive morphological filter was proposed. Firstly, a set of structuring factors was constructed to detect different sudden changes of pixel values in images. Then a corresponding structuring element for morphological filtering was established for each structuring factor. Finally, the most possible structures of sudden changes in each pixel’s neighborhood were detected by using the set of structuring factors and a morphological filtering was performed to the pixel with the corresponding structuring element. The comparison experiment was carried out on simulated and real ultrasound images by using the proposed method, the speckle reducing anisotropic diffusion and different scale morphological filters, respectively. It is shown that the proposed method averagely improves the Signal-to-noise Ratio (SNR), Contrast-to-noise Ratio (CNR) and Figure of Merit (FOM) of ultrasound images by 15%, 37% and 69%. Therefore, the adaptive morphological despeckling filter may be an effective method to suppress speckle in ultrasound images.