Aiming at the degradation of actual reconstructed image quality caused by the mismatch between ideal and practical optical transmission models in a computer-generated holography algorithm, this paper proposes a simplified learning-based holographic optical transmission model that uses physical information. The proposed model can explicitly learn the defects of holographic displays and can be flexibly used in various hologram optimization algorithms to solve the mismatch problem in optical transmission models. In the untuned holographic display prototype, reconstruction results of the proposed model are superior to those of ideal holographic light transmission model. Moreover, the proposed model can obtain high-quality holographic reconstruction images without strict requirements such as the fine assembly of optical elements and the good uniformity of laser light sources.