The radial velocity method is quite effective in discovering and characterizing exoplanets based on the radial velocity change of the target stellar. It plays an important role in the detection of exoplanets. Owing to stellar activities, the differences between spectra and template, and the noise in spectra for other reasons, radial velocities calculated by the cross-correlation function algorithm may have some errors. This paper proposes a method of measuring stellar radial velocities, and an attention mechanism is integrated. Observed spectra are processed to remove noise from the spectra, and radial velocity is calculated based on the periodicity of stellar radial velocity. First, Gaussian Process Regression is used to establish spectral models, which helps reduce the influence of noise and get more precise spectra. A subset of data is used to reduce the computation cost. Then, based on the idea of attention mechanism, different weight is assigned for the absorption lines to figure out differential radial velocities between spectra. Finally, radial velocities of stellar are obtained using the relationship of differential radial velocities. This paper analyzes the effect of the signal-to-noise ratio and the number of spectra on the mean error of radial velocities. The experimental results show that compared to the cross-correlation function algorithm, the stellar radial velocity calculation method combined with the attention mechanism can effectively reduce the error of radial velocities when the signal-to-noise ratio is low. Increasing the number of spectra helps improve the accuracy of radial velocities to some degree. Finally, The spectra of HD85512 are analyzed. Compared with other algorithms, the algorithm proposed in this paper reduces the error of radial velocities and greatly improves the accuracy.