In the nonuniformity correction problem of infrared focal plane array detector (IRFPA), the traditional neural network algorithm will appear the image edge blur, low contrast, ghosting artifacts and other phenomena. Aiming at these phenomena, this paper proposes an improved neural network nonuniformity correction algorithm based on side window filtering. The algorithm first uses side window filtering on the input image to obtain the desired image, and protects the edge details of the target while removing the non-uniform noise to improve the image quality. On this basis, it can effectively avoid the ghosting artifacts problem of the corrected image by suppressing the local divergence of the correction parameters through the saturated nonlinear function. The experimental results show that the algorithm proposed in this paper can effectively remove the non-uniform noise in the image, and there is no obvious ghosting artifacts phenomenon. The average image roughness of the three groups of test image sequences is reduced by 30.17%. The maximum time consumption for continuous processing of 400 image sequences on the experimental computer is 37.417 0 s, which is 95.05% less than that of the comparison algorithm improved based on bilateral filtering, and 45.81% less than that of the comparison algorithm based on wavelet principal component analysis. The algorithm in this paper has obvious advantages in nonuniformity correction effect and algorithm operation efficiency, which provides a new research idea for real-time nonuniformity correction on mobile platforms with small computational power and low power consumption.