Segmentation of whole brain regions has important clinical significance for the diagnosis and treatment of brain diseases. In order to improve the accuracy of segmentation, a multi-atlas segmentation algorithm for polynomial expansion registration is proposed. Firstly, by using the model of linear polynomial expansion and combining affine transformation and non-rigid body transformation, the target image to be segmented and the map image are registered one by one to obtain the displacement field. Then, the similarity between the target image and the atlas is calculated by normalized mutual information, in which the atlas with high similarity to the target image is screened out. Then, the marked image of the atlas is mapped by displacement field, and the rough segmentation result is obtained. Finally, the global weighted voting method is used to fuse the obtained coarse segmentation results to obtain the final fine segmentation results. 35 T1-Weighted MRI images from the MICCAI 2012 multi-atlas Labeling challenge and 10 T1-Weighted MRI images from Shanghai Ninth People’s Hospital, Shanghai Jiao Tong University School of Medicine are selected to verify the method. Finally, DSC values of the whole brain region are 0.7585 and 0.7351. Experimental results show that the algorithm has high accuracy and robustness in brain segmentation, which is expected to assist clinical diagnosis and treatment of brain-related diseases.