To improve the precision of an opto-electronic tracking system without installing new equipment, an assistant tracking with surplus angle information was carried out and a filter tracking only by angle information was designed. Firstly, the working principle of the opto-electronic tracking system with a laser-infrared detector was introduced, and a target model based on the “current” statistical model was established in a globe-coordinate. Considering the complex noise interference in the high frequency image series in a angle track process, the set-membership estimation was introduced to repair the maneuvering model, then a self-adjusted ellipsoidal bounding filter whose tracking was only by angle was designed. Finally, a simulation on the course tracking to enemy targets was performed by the opto-electronic system on the ship. The comparison results show when the laser detector probability decreases to fifty percent, the filter designed by ellipsoidal bounding algorithm could track the maneuvering target stably with a Room Mean Square Error(RMSE) of target angle less than 06 mrad.The filter could overcome the effect of non-white noises and has some practical values in engineering.