Using the YOLOv3 architecture, we propose a recognition method for fast low-altitude unmanned aerial vehicle (UAV) detection based on the dual channel (Dual-YOLOv3). In this method, the infrared and visible UAV images are simultaneously input into the deep residual network for feature extraction, and the extracted feature maps are fused to enhance the expression ability of the features. Then, the multi-scale prediction network is used to determine the classification and the position regression of the UAV targets. Finally, we obtain the detection and recognition results. Comparison experiments are conducted on the real collected dataset of dual-band UAVs. The results show that the mAP (mean of average precision) of Dual-YOLOv3-D with average fusion is improved by 6.1% as compared with that of YOLOv3 with the single data source; the detection speed is approximately 27 s ^-1.