The cavity increases the rotation angle by increasing the length of the interaction between probe light and atoms, which greatly improves the sensitivity of a magnetometer. On an atomic magnetometer based on the non-linear magneto-optical effect, we have theoretically and experimentally studied the relationship between the enhancement factor of the sensitivity and the reflectivity of the cavity mirror, and obtained the optimal sensitivity with certain loss of probe light in atomic vapor cell. In a plano-concave standing-wave cavity, when the reflectivity of the input mirror and output coupling mirror is respectively 99.5% and 88%, the loss of the light intensity of the atomic vapor cell is 9.5% and 8.5sdown,thesensitivityofthemagnetometerisimproved,andthecavityenhancementeffectismoresignificant.