Edge extraction algorithms in digital image processing are often noise sensitive in detecting building cracks due to un-uniform illumination and the surounding affects. An accumulated gradient algorithm was presented to calculate the crack boundary. An adaptive binarization algorithm and morphology operations were adopted to obtain an image mask firstly. The mask image was acted on the original image to obtain the expanded image of the crack area. Then the image was iteratively expanded, and the gray value of the image was added to obtain the cumulative gray value.The accumulated gray between two successive images before and after expansion were subtracted, and the maximum difference was the cumulative gradient, the image before expansion was considered as the extracted crack. Experimental results show the crack can be extracted by this algorithm in complex backgrounds.