Since the invention of holography by Dennis Gabor, the fabrication of holograms has mainly relied on direct recording of wavefront by engraving the intensity fringes of interfering electric fields into the holographic material. The degree-of-freedom (DoF) is often limited, especially for its usage as a holographic optical element in imaging or display systems, as what is recorded is what to use. In this work, based on the emerging self-assembled photo-aligned liquid crystal, a polarization hologram with full DoF for local manipulation of optical structure is demonstrated. The ability to record an arbitrary wavefront (in-plane DoF) is achieved by freeform surface exposure, while the local adjustment of deposited liquid crystal (out-of-plane DoF) is realized by inkjet printing. The methodology for designing and fabricating such a hologram is exemplified by building a full-color retinal scanning display without color crosstalk. Here, the arbitrary wavefront modulation capability helps to eliminate the aberrations caused by mismatched exposure and display wavelengths. The local liquid crystal adjustment ability enables the suppression of crosstalk by variation of chiral pitch and film thickness to tune the peak and valley of Bragg diffraction band. The demonstrated method is expected to greatly impact the fields of advanced imaging and display, such as augmented reality and virtual reality, that require optics with an ultrathin form factor and high degrees of design freedom simultaneously.