In this study, a regularization method of point spread function (PSF) estimation and image reconstruction considering multiple blur factors about image degradation is proposed for wide-field polarization-modulated microscopic imaging. The adaptive regularization model of variable exponential function is used for the PSF estimation, which is degraded due to the fitting deviation of polarization-angle modulation curve, the degradation of optical system, and discrete under-sampling of charge-coupled device (CCD) during the blurring process. The aim is to fully utilize the acquired image content features, adaptively select a variable regularization norm, and effectively restrain the staircase effect of total variation regularization and the disadvantage of poor edge-preserving property of Tikhonov regularization. The optimized Split-Bregman iterative algorithm is adopted in the solution process, which can ensure the estimation accuracy and reduce the computational complexity as well. Experimental results show that the proposed method can effectively estimate the degraded PSF and improve the noise robustness of image reconstruction.