In order to realize the multi-target tracking of pig’s behavior in pigsty complex light situations and shelter among pigs, the traditional multi-object tracking algorithm is improved, and a target tracking system based on computer vision technology is developed. The foreground detection algorithms are improved, and the difference images obtained by gray difference, S-channel difference and frame difference are fusion. The color differences and texture similarity between shadow and background are used to eliminate the effect of shadow upon detection. The camshift algorithm is improved to realize the accurate tracking of pig's motor behavior in pigsty. The improved algorithm integrates the initial position and color information of target pigs, and fixes the histogram model. It improves the tracking accuracy. By the experiments of tracking different varieties pigs and tracking under different light situations, the experimental results show that the proposed target detection method can effectively eliminate the influence of the shadow and the experiments verify the stability of the algorithm, the tracking accuracy rate is greater than 85%.