Based on the efficient FAST (features from accelerated segment test) algorithm, a color based-FAST (C-FAST) algorithm for image feature detection and matching with color information improvement is proposed. The algorithm adds scale invariant and rotation invariant feature descriptors which the original FAST algorithm does not have, and takes color information as an important reference factor in feature detection and matching. Hence, the proposed algorithm is more efficient and has higher detection and matching accuracies. It also has good robustness under the conditions of illumination changes and noise effects. Different algorithms like FAST, speeded up robust features (SURF), colorful scale-invariant feature transform (CIFT) and the proposed algorithm are analyzed via public data sets and common images. The running data prove that the proposed algorithm can detect and match the image features effectively and reliably, with 30% accuracy improvement compared with the original FAST algorithm.