The aim of this paper is to address the difficulty in pedestrian target recognition in intelligent assisted driving systems due to the influence of light and climate on visible light cameras. A pedestrian target detection algorithm is implemented and improved by studying image fusion techniques in combination with deep convolutional neural networks. Firstly, using multi-source sensor image fusion technology, the strategy of fusing visible light cameras and infrared thermal imaging cameras, based on the Faster RCNN algorithm, a pedestrian target detection algorithm based on infrared thermal imaging technology and improved depth convolutional neural network is proposed. Then, the research is carried out in terms of improving network structure, feature fusion, optimising model training, and so on, and the research is carried out on pedestrian detection and localisation tracking in complex environments. Finally, the experimental results show that this algorithm improves detection efficiency and accuracy for human target detection in complex climate environments, and increases the safety of intelligent assisted driving vehicles.