Current ellipse detection algorithms have been designed and implemented for grayscale images. When processing color images, much useful information is discarded, which is inconducive to obtaining higher quality detection results. Therefore, an ellipse detection algorithm for color images is proposed. First, the three color channels and weighted grayscale image of the image are detected for ellipses, and a set of ellipses is obtained by fusing the detection results of multiple channels. Then, the multiple response results of the same ellipse in different image channels are clustered and combined into an ellipse. Finally, an ellipse validity verification technique is proposed. The technology fuses the color information of the image using the DiZenzo operator, extracts the ellipse support line segment to determine the effectiveness of the ellipse, filters invalid ellipses, and obtains the final detection results. Experiments show that compared with existing algorithms, the proposed algorithm fully uses the color information of the image, thereby significantly improving the efficiency of ellipse detection, and the F-score on a standard dataset is significantly better than that of the current detection algorithms.