Because of the application requirements of high image quality, full-color display, and the simple and compact structure of a helmet-mounted display system, a dual-free-form prism structure is proposed. A dual-channel helmet-mounted display optical system is designed using the off-axis folding/reflection principle. Furthermore, a dual-prism model is constructed using the custom constraint function, and the off-axis non-rotational symmetry system is optimized using the rectangular array pupil sampling algorithm, which is combined with vector aberration theory. In the design of the projection and transmission dual-channel system, which has a working wavelength of 400-700 nm, the diameter and distance of the pupil are 8.5 mm and 21 mm, respectively, and the volume is 41.5 mm×31.5 mm×14.1 mm. The field of view of the projection channel is 45°, and the full field of view modulation transfer function value is greater than 0.3 at 61 lp/mm. The transmission channel field of view is 52°, and the full field of view modulation transfer function value is greater than 0.4 at 30 cycle/(°). The dynamic image quality analysis of the dual-channel system is conducted, and the analysis results show that the image quality received by the human eye is unaffected by the attitude of the eyeball within the range of the diameter of the pupil. This dual-prism structure provides a reference design direction for the next generation of the dual-channel helmet-mounted display system.