As the traditional segmentation algorithms are difficult to realize accurate segmentation for high resolution multispectral image , this paper proposed a kind of fuzzy clustering segmentation algorithm of multispectral image on the basis of Gaussian mixture model(GMM). GMM was adopted to define the dissimilarity measure of pixels to the clusters.As the proposed algorithm has the ability of high-precision fitting data statistics distribution,it can effectively eliminate the negative impact of noise on segmentation results. Hidden Markov Random Field (HMRF) was brought in to define prior probability of neighborhood relationship simultaneously,then the prior probability was used as weight of each Gaussian component and parameter to control cluster scale in Kullback-Leibler (KL), so the robustness of algorithm to remote sensing image at complex scene was increased and segmentation accuracy of algorithm was further improved. Qualitative and quantitative analysis were conducted on segmentation result of simulated image and high resolution multispectral image. Experimental result shows that the total accuracy of simulated image exceeds 96.8%, which verifies that algorithm mentioned has detailed information keeping capacity when performing segmentation to high resolution multispectral image and verifies effectiveness and feasibility of algorithm. The algorithm can realize accurate segmentation of the high resolution multispectral image.