In order to solve the problem of aberration detection failure caused by double-layer reflected light of the fundus retina in standard animal model mouse during wavefront detection, a mouse eye aberration measurement technique combined with optical mask modulation was proposed to improve the accuracy of wavefront aberration measurement. First, according to the key parameters of mouse retina, we established the optical system model of mouse eye wavefront aberration detection and performed optical simulations. Then, the effects of optical masks with different apertures on the reflection beam of the non-target layer of the retina were analyzed and compared, and then the parameters of the optical mask and the experimental plan were determined. Finally, the wave front aberration detection system of the mouse eye was established, and the wavefront aberration of the mouse eye was measured in vivo. The experimental results show that the optical mask with 0.5 mm aperture can reduce the root mean square error of mouse eye wavefront aberration measurement by 74.9%, which is similar to the shielding effect of non-target layer reflected in 80% of the theoretical simulation. It can effectively block the reflected light from the non-target layer of the mouse retina, improve the detection accuracy of the wavefront aberration of the mouse eye, and lay a foundation for the further realization of high-resolution imaging of the mouse eye.