In order to improve the effect of CNN feature driven flotation performance recognition under small-scale training set， a method of flotation performance recognition based on adaptive transfer learning and CNN features extraction of foam infrared and visible images is proposed. Firstly， a dual-modality CNN feature extraction and recognition model based on AlexNet was constructed， and the structural parameters of the model were pre-trained through RGB-D large-scale data set. Secondly， a series of double hidden layer automatic encoder extreme learning machine is used to replace the full connection layer of the pre-training model， so that the dual-modality CNN features can be fused and abstracted layer by layer， and then the decision is made by mapping to higher dimensional space through the kernel extreme learning machine. Finally， the floatation small-scale data set is constructed to train the migrated model， and the improved quantum wolf pack algorithm is used for model parameter optimization. Experimental results show that， adaptive transfer learning can significantly improve the accuracy of recognition in small sample data sets， the accuracy of performance recognition using dual-modality CNN transfer learning is 3.06% higher than that of single-mode CNN transfer learning， and the average recognition accuracy of each working condition reached 96.86%. The accuracy and stability of flotation performance recognition is greatly improved compared with the existing methods.