As Scale Invariant Feature Transform(SIFT)describes local characteristics of images only and ignores the color information of the images, it has higher match errors when a lot of similar regions in the images are matched. This paper improves the SIFT algorithm and proposes a novel method as an extension of the SIFT, called a Shape-color Alliance Robust Feature (SCARF) descriptor, to resolve the problems mentioned above. The proposed approach SCARF uses the SIFT descriptor to extract the feature point set of the images. Then, by building a concentric-ring model, it integrates a color invariant space and a shape context with the SIFT to construct the SCARF descriptor, and uses the Euclidean distance as cost function to match the descriptor. A comparative evaluation for different descriptors is carried out by the INRIA database, which verifies that the SCARF approach provides better results than other four state-of-the-art related methods in many cases, such as viewpoint change, zoom+rotation, image blur and illumination change. It concludes that the SCARF reduces the probability of mismatch and improves the stability and robustness of matching process greatly.