Binocular Multi-FOV Sun-photometer (BMFOVSP) is used to measure the optical properties of cirrus. In order to solve the localized problem of traditional sun-photometer tracking sun inaccurate by using four-quadrant detector when the cloud is in homogeneous, image tracking system of BMFOVSP is developed by using image processing technology to track sun. The all-weather and full automatic sun tracking in the absence of drizzle is achieved by adopting the double modes tracking technology with the combination of both calendar-check tracking and image processing technology tracking. The tracking accuracy is higher than one minute of angle. The system fits BMFOVSP for measuring the optical properties of cirrus. It also can be applied to other occasions which require high sun tracking accuracy. Gross structure, system principle and the design of imaging unit are introduced. The methods of both calendar- check tracking and image tracking are represented in detail. The reliability and stability of system are analyzed. The graphs of tracking effect are shown, which means the proposed tracking system can track sun primely and have high tracking accuracy.