To address the high sensitivity of polarization images to noise and the difficulty of conventional polarization image denoising techniques in ensuring accurate restoration of polarization information while eliminating noise, a 4D block-matching (PBM4D) polarization image denoising algorithm based on polarization constraints is proposed. Initially, the algorithm leverages Stokes relationships to enhance the polarization image and transform it into 3D data across the polarization dimension. Subsequently, 4D block matching is employed based on the similarity of polarization information to ensure fidelity in preserving polarization details. Additionally, the algorithm capitalizes on the physical correlation between distinct polarization channels. Finally, collaborative filtering in the 4D transform domain is applied to suppress noise effectively. Comparative analysis demonstrates the superior noise reduction capabilities of PBM4D across various signal-to-noise ratios, particularly evident in linear polarization degree and polarization angle images. Importantly, PBM4D retains the physical correlation and inherent properties of polarization images, facilitating robust recovery of polarization information from objects exhibiting diverse polarization characteristics.