The traditional visual attention model for the detection of visual saliency regions in the remote sensing image can lead to high computational complexity and low precision of detection. A new fast detection algorithm of visual saliency regions is proposed. The new algorithm firstly decreases the spatial resolution by integer wavelet transform, which can reduce the computational complexity of detection of the visual focus of attention. Then, the new algorithm proposes the two-dimensional discrete moment transform for visual feature fusion, which can generate the saliency map of the remote sensing image which has more abundant information of edge and texture. Finally, the region growing strategy based on the visual focus of attention is proposed in the saliency map analysis to acquire the precise contours of the visual saliency regions. The experimental results show that the new algorithm can not only effectively reduce the computational complexity of the detection of visual saliency regions in the remote sensing image, but also be able to accurately describe the contour information of visual saliency regions. In addition, it can avoid image segmentation and feature extraction for the whole image. The new algorithm provides a certain reference for the target detection of the remote sensing image in the future.