The problem of shape target detection was formulated as a binary classification problem of each pixel under Markov Random Field (MRF)theoretical framework and the adaptive neighborhood system of MRF was introduced. Firstly, the factors that cause the changing of target shapes were analyzed and classic target shapes presented on obtained infrared images were concluded. Secondly, the classic shapes were used as templates while establishing the new neighborhood system of MRF. Thirdly, to achieve the optimal detection performance, a criterion function for adaptively selecting the proper neighborhood for each pixel was proposed and at last a new potential function using finite difference operator was proposed for the classification of target and background at each pixel. For the usage of adaptive neighborhood system, the proposed algorithm has following advantages: further reduction of the threshold crossing rate of target detection in single image frame while maintaining the target detection rate and better preservation of target shape details than algorithms using classic neighborhood system of MRF. By simulations and experiments, the results show that the proposed algorithm can optimally detect targets under various image Signal-to-clutter ratios, and perfectly protect target shape details.