A blob detection algorithm based on multi-scale second-order anisotropic Gaussian directional derivative filter was proposed to address the shortcomings of current blob detection algorithms in locating and describing blob shapes. The second-order anisotropic Gaussian directional derivative （SOAGDD） could effectively extract image information from different directions and cope with noise interference. In this paper， a multi-scale second-order anisotropic Gaussian directional derivative filter was used to smooth the input image and calculate the regional extremum of the processed image. The experiments have demonstrated the superiority of our proposed method in the localization of interest point detection， shape description of detected interest points， and image matching.