There is a contradiction between the 14 bits output data of an infrared thermal imager and the 8 bits input data of its monitor in a real-time infrared image processing system. A novel algorithm for real-time infrared image enhancement based on Field Programmble Gate Array(FPGA) is proposed not only to solve this problem but also to satisfy the real-time and mass data needs of the system. Firstly, an image mean filter is used to compute the local limit values for each frame dynamically. Secondly, a multi-level average statistics method is designed to calculate the values of segment points for each gray normalization partition. Finally, the mean of different linear normalizations on different gray level segments is calculated to enhance the infrared image. The whole algorithm is developed on FPGA hardware with Verilog language. Experimental results show that this algorithm resolves the problem that the output values of an un-cooled focal plane array detector vary with its working life. Moreover, the algorithm reduces the impact from bad pixels, and enhances the detail of an image that has objects at both high and low temperatures in a scene. Compared with histogram projection, one of the most traditional image enhancement methods, this algorithm shortens the computation time by about 1.38 ms and the memory resource reduces by 97.6%. It can be implemented more easily by FPGA and performed better in a real-time infrared image enhancement system. In terms of engineering applications, the proposed scheme with simple architecture, high reliability and low cost characters can provide a valuable reference for future designs of real-time infrared image processing systems.