The detection and recognition accuracy of deep learning algorithm based on convolutional neural network is greater than the traditional pattern recognition algorithm, but the convolution and nonlinear activation function of neural network takes a lot of computing power to do it efficiently, this makes it hard for a lot of deep learning algorithm model in power limit embedded platform for deployment. In this paper, the target detection algorithm YOLO-V3 is taken as an example, and the corresponding FPGA implementation method is designed for different layers of the network, and the parallel operation element based on tiles is designed especially for the convolution layer. Finally, a target detection hardware accelerator is realized in FPGA. The accelerator can make full use of the hardware computing resources of FPGA, and its overall average performance is 192.229 GOP/s. Through experimental comparison, it is proved that this paper can provide a solution of high energy efficiency and high recognition accuracy for embedded infrared target recognition system.