For the problem of a few feature points and low registration accuracy in multi-mode magnetic resonance imaging (MRI) tumor image registration by speeded up robust features algorithm. Harris detection method is used to extract and detect the feature points of reference image and floating image, and then circular 64-dimensional vector method is used to generate feature descriptors and carry out Euclidean distance matching to enhance the extraction of feature points. The initial parameters of registration image are set to ensure that particles search near the optimal value. The mutual information is used as the measure function of particle swarm optimization to enhance the global optimal solution of the objective function. The average maximum value is introduced to prevent this algorithm from falling into premature phenomenon. Simulation results show that compared with the existing algorithms, the proposed optimization algorithm can increase the number of multi-mode MRI image feature points with higher accuracy.