To improve the high false-alarm rate and poor real-time capability in detecting infrared small dim targets, a novel algorithm based on visual saliency and local entropy is proposed in this paper. This method solves the problem from coarse to fine detecting of small targets. First, a local entropy method is used to obtain the region of interest. Then, an improved visual saliency method is used to calculate local contrast. Finally, a threshold segmentation method is used to extract dim infrared small targets. The method is verified using a contrast test with TOPHAT and LCM, and the results show that the performance of this method precedes the TOPHAT algorithm and LCM algorithm. The false alarm rate by this method decreases to 62.5% and 33.3% compared with the other two algorithms, and the time cost decrease to 38.6% of that of LCM. The method can achieve accurate detection of infrared dim and small targets in a complicated environment, solving the high false alarm rate and poor real-time capability issues to some extent.