In order to improve the visual effect of low illumination video, the high-quality daytime information of the same scene is often introduced to enhance nighttime video. A novel frame illumination-based method of video enhancement is proposed. In order to require daytime background, we firstly use averaging K frames method. Then, illumination of daytime background and nighttime video frames is extracted by using Retinex theory to enhance nighttime motion objects. Moreover, nighttime video frame’s motion objects are extracted by using different frames method. Lastly, daytime illumination is used to fuse nighttime video problem of the same scene between daytime and nighttime videos. For improving the perception qualities of the moving objects, we further propose Gaussian filter method for nighttime video enhancement which is efficiently and smoothly moving objects to the reasonable enhancement results. Experimental results show the effectiveness of the proposed algorithm can improve visual quality over conventional methods.