Traditional moving target detection under a moving base has a problem of larger background interference, and its detection accuracy is effected by the noise interference. This paper proposes a moving target detection method under the moving base by using orthogonal Fourier transform and kernel-grayscale chart to overcome the influence of a larger illumination change and background noise on moving target detection accuracy. Firstly, the evaluation function was introduced the selection of a feature matching block to implement the sub-block matching of video backgrounds. Then, global motion compensation parameters were estimated by using a phase-correlation algorithm based on orthogonal Fourier transform, and each Gaussian kernel value of each sub-block of the image was calculated to build a nuclear function-gray chart and to determine the area of moving target according on the change of the adjacent frame Gaussian kernel value. Finally, the image sub-block with moving target was divided and processed, and the moving target detection was implemented. The simulation in comparison with conventional moving object detection algorithm shows that when the evaluation coefficient in the evaluation function is set to be 0.7, and inter tile similarity threshold to be 0.3, the algorithm can effectively inhibit the background interference from illumination changes and background noise and can detect the moving target under the moving base. The algorithm has fast calculation speeds and meets real-time requirements of engineering.