Because of the influence of surface objects, such as clouds and birds, etc., the false alarm rate of infrared surveillance system is easy to be higher than the imagined. In order to decrease its false alarm rate effectively, a detection algorithm is researched by analyzing the changes of object’s angle coordinate. An optimal method is used which combines the interior point penalty function method and the steepest descent algorithm, so as to judge the routes of all the correlative objects and to differentiate the real object from all the interferences. At last, some experiments were carried out to prove that the proposed method can improve the anti-interference capability and decrease the false alarm rate in real time.