Because of large amount of calculation, it took a long operating time for the tensor vote algorithm to restore human broken target in infrared images. Thus it can't meet the need of real-time pedestrian detection. An improved tensor voting algorithm is researched. Firstly, most of the points beyond the vote field are filtered out by converting the strength threshold into distance threshold. Then, according to symmetry of two voters’ voting effect to each other, it’s not necessary to compute strength value again. Finally, we use the strength threshold to precisely filter out the points beyond the voting field. Simulation results show that the improved tensor voting algorithm can reduce calculation cost and shorten running time, without loss of the restoring effect.