SignificanceProgress in imaging and display optical systems exerts significant influences on the development of science and technology. Imaging and display systems intrinsically utilize optical elements (geometric or phase elements) to modulate optical wavefronts and achieve expectational imaging relationships, system specifications, and structure requirements. As the representative elements of geometric and phase elements respectively, freeform optical elements (FOEs) and holographic optical elements (HOEs) have significant advantages in optical system design. FOEs possess high degrees of design freedom, which can greatly enhance the ability to modulate wavefronts and improve imaging performance. Additionally, freeform surfaces can correct the aberrations of optical systems with off-axis nonsymmetric structures. Meanwhile, HOEs can unconventionally deflect rays at large angles due to their unique ability to modulate optical wavefronts. They can dramatically reduce the weight and volume of optical systems due to the lightweight form factor, and realize better optical see-through experiences and full-color display due to unique selectivity and multiplex ability, achieving mass productions owing to relatively simple fabrication methods and low costs. Meanwhile, it is easy to fabricate HOEs with large sizes due to the unique fabrication methods. Considering the above-mentioned advantages, designers may design imaging and display optical systems that combine FOEs and HOEs, significantly improving the degrees of design freedom and the ability to correct aberrations. Additionally, we can achieve advanced system specifications, excellent system performance, compact and lightweight system forms, and unconventional system structures with off-axis nonsymmetry, with further development of optical systems promoted. It is important to summarize the existing design methods of imaging and display systems combining FOEs and HOEs, analyze the problems restricting their further development, and predict the development trends. Meanwhile, it is essential to summarize the existing designs and applications of these systems to better guide and promote the development.ProgressWe describe the basic principles, ray-tracing models, advantages, and applications of FOEs and HOEs respectively, summarize the system design methods, review the designs and applications of these systems, and analyze current restrictions and future development trends. The design of these systems can be divided into three types. 1) FOEs and HOEs are simultaneously utilized to correct the aberrations of optical systems. 2) The freeform surface is adopted as the substrate shape of HOEs. 3) During HOE fabrication, FOEs are introduced to modulate the recording waves of HOEs. In practical optical system designs, the design can be a combination of the above three ways. The first way directly builds ray-tracing models of freeform optics and HOEs in the optical system design and then adopts the optimization strategy to achieve expectational requirements. The second way coats the holographic recording medium on the freeform substrate to yield HOEs with freeform substrates. The third way bridges the numerical relationship between freeform optics and recording waves of HOEs to fabricate HOEs with unconventional profiles of holographic phase function or grating vector. The methods for defining HOEs based on ray tracing are described in detail, including the phase functions (direction cosines) of the recording waves, holographic phase function, and holographic grating vector, which guides the basic combined design schemes. We review the ways of fabricating HOEs including the whole-area exposing and sub-area exposing (holographic printing) to provide references for combined design fabrication. The calculation methods of starting points of optical systems based on HOEs are summarized in detail, including point-by-point construction and iteration methods, confocal methods, and simultaneous multiple surface (SMS) methods, which guide the design of the optical system combining FOEs and HOEs. The designs and applications of these systems are summarized based on the classifications of HOEs, including augmented reality (AR) near-eye display systems, head-up display (HUD) systems, and HOE-lens imaging systems. Additionally, combined designs of freeform optics and other types of phase elements are also presented, such as liquid crystal polarization hologram (LCPH) based on freeform exposure, and metasurfaces with freeform substrate, which has certain guidance for the combined design of FOEs and HOEs.Conclusions and ProspectsStudies on the system design combining FOEs and HOEs make significant progress in the basic principles, design frameworks, and fabrication methods, which has been employed for developing imaging and display systems with high performance, novel structure, and lightweight form factor. There are also some problems and challenges for the research on the system design combining FOEs and HOEs. They include how to fabricate HOEs with freeform substrates by innovative coating technologies of the holographic recording medium, how to correct chromatic aberrations in the imaging and display system using HOEs, how to reduce the nonuniformity of diffraction efficiency and stray light of systems combining FOEs and HOEs, and how to conduct tolerance analysis of such systems. In summary, the research on the design of imaging and display systems combining FOEs and HOEs will promote the development of next-generation high-performance and compact optical systems.