In order to realize three-dimensional (3D) reconstruction for canopy and guide the operations of intelligent pruning, flower thinning and picking, a multi-image registration algorithm of apple tree at different growth stages is restigated. The robot vision system includes a color camera system and a photo mixing detector (PMD) camera. Image features at different growth stages are described by image similarity and reliable pixels. According to these image features. The optical calibration based on the distance information can be used from the flowering to the mature stage, while from the dormancy to the germination stage, the multi-image registration algorism should be used. Ninety groups of pictures, including the stages of mature, dormancy and flowering are used to verification the viewpoint. The experimental result shows that, the matching rate reaches 100% at mature stage and flowering stage, while 86.11% at dormancy stage.