Urban drainage system is a crucial part of urban public facilities; thus, the regular inspection and maintenance of drainage pipe is essential for safely operating underground pipe network. The drifting capsule robot developed by the project team is characterized by its convenient operation, high efficiency, and inexpensiveness, which meet the requirements of large-scale survey of underground pipe network. However, the high-efficiency operation mode results in a huge amount of data that needs to be processed. Simultaneously, video data collected by drifting operation contains several unwanted features, such as vibrations and illumination, thus traditional data processing methods are unsuitable. Therefore, there is an urgent need to develop new intelligent disease recognition methods. This study presents a disease identification method based on an improved residual attention network. This method considered video clips as input, used convolutional neural networks to extract the features of each frame, and then fused different layers along specific dimensions for classification and recognition. Experimental results show that the improved method can achieve an accuracy of 89.6%, better than unimproved residual network, and effectively improve the recognition accuracy and efficiency of the drifting capsule robot.