Aiming at the problem of infrared remote sensing cloud detection under complex surface features background, a novel cloud detection algorithm based on image feature extraction, regional voting and threshold segmentation is proposed. Denoising and normalized stretching are performed on image. The multi-features are extracted and cloud discrimination is executed in sub-region using the extracted feature vectors. The precise cloud regions are obtained through threshold segmentation based on discrimination results and fractal characteristics. Test results show that the accuracy of image recognition is above 91%. The applicability and effectiveness of the algorithm is verified by large quantities of measured data and it provides effective technical support for the information processing of infrared remote sensing image.