Division of focal plane(DoFP) polarization imaging detector are composed of integrated micro-polarizer array on a focal plane array sensor, which make the DoFP polarimeters capture the polarization information real-time. However, it is difficult to perform the DoFP demosaicking and reconstruct the polarization information due to noise. A PCA-based spatial-temporal adaptive denoising method was presented to work directly on the DoFP videos. For each DoFP patch to be denoised, similar patches were selected within a local spatial-temporal neighborhood. The principal component analysis was performed on the selected patch to remove the noise. The spatial-temporal information of DoFP video was used to construct the sample patches. The proposed method worked directly on the DoFP video without explicit motion estimation. And then a fast bilateral filtering algorithm was used to remove the residual noise in different polarization channels of DoFP images. The experimental results on simulated and real noisy DoFP sequences demonstrate that the proposed denoising method can significantly reduce the noise-caused polarization artifacts and outperform other denoising methods.