Head-mounted three-dimensional (3D) display device is a common presentation carrier of human-computer interaction interfaces in the fields of virtual reality, augmented reality and metaverse. Currently, the image quality evaluation methods and systems for head-mounted 3D display are insufficient. A 3D image quality evaluation theory based on the human visual system was proposed, and a 3D image quality evaluation method and the corresponding system were designed. Some parameters including depth reconstruction, distortion, ultimate resolution, and field angle were quantitatively evaluated. The depth reconstruction was tested based on binocular vision principle, the distortion was expressed by curvatures and polynomial transformations of specific points, the ultimate resolution was evaluated by MTF and valley-to-peak ratio of patterns, and the field angle was measured based on pinhole imaging method. In the test of two head-mounted display devices, the relative error of depth reconstruction is smaller than 3.5%, the distortion is smaller than 3%, and the field angle are 77°1′44″ and 86°56′26″, respectively. The evaluation results are highly consistent with the subjective perception of the human eyes. The proposed method and system have high engineering applicability, which are expected to play an important role in the quality assessment and design improvement of the head-mounted 3D display.