A multiple hypothesis method for tracking multiple motion objects is proposed based on the object detection,which is robust to changes illumination and shadow conditions in image sequence. After video preprocessing including background suppression,temporal filter and clutter removal,the initial results of object detection is obtained in each frame image. Based on the detection results which may have missing and/or false detection,the multiple object tracking algorithm accepts the probabilities of initial object detection and keeps multiple hypotheses of object in a graph structure. Through extending and pruning the graph based on the previous image information,the multiple object tracking method gives feedbacks which are predictions of object locations to the object detection module. Therefore,the method integrates object detection and tracking tightly. The most possible hypothesis provides the multiple motion object tracking results. The experiment and evaluation results illustrate the validity and advantage of the proposed algorithm.