At present, there is little work in the field of dehazing that introduces prior information into data-driven deep learning methods, and most dehazing networks based on deep learning usually have high requirements on computer memory and computing power. To solve the above problems, this paper proposes a multi-scale fusion dehazing network for high-frequency information alignment (HFMS-Net). The network framework adoptes a cycyle pattern: for the generator, residual connections are introduced at different depths of the lightweight convolutional neural network to make full use of the intermediate layer features of the network to achieve multi-scale feature fusion; for the discriminator, the network needs to extract texture information on its input to approximate the high-frequency information between the dehazed image and the hazy image, making the data-driven network more physically interpretable. Compared with PFDN, HFMS-Net achieves superior performance with about 1/5 of the memory footprint under the same setting, and the PSNR and SSIM are improved by 0.71 and 0.016, respectively. Through a large number of comparative experiments and ablation experiments, it is proved that the dehazing performance of this network has a certain improvement compared with the existing algorithms, and higher fidelity to texture information.