In the complex infrared scene of the battlefield, the target is easily confused in the background due to the irregular gray level distribution, the blurred edge of the object and the lack of texture features. Due to the limitation of the computing performance of the embedded platform, most of the deep learning detection algorithms are difficult to apply to portable devices and cannot achieve fast and effective object recognition. An object recognition method based on moving object extraction and efficient machine learning model is proposed. This method firstly realizes the target pixel-level segmentation through motion detection, and locate the single target after morphological processing. Then select lightweight deep network features or contour features according to the computing performance of the embedded platform, train softmax model to achieve object classification. This algorithm is transplanted to the embedded platform, and the object recognition experiment is performed on the open source infrared image sequence, which can realize the simultaneous positioning and classification of multiple objects, and the processing speed is up to 56FPS. Experimental results show that this method can effectively identify infrared targets in complex backgrounds in real time.