In order to solve the common problems of current medical images, such as relatively scattered size distribution of pathological areas, the ambiguous detail features, and the big visual difference of similar tissue images, this paper proposes a medical retrieval method integrating multi-scale features and attention mechanism based on the CBMIR system. This method adaptively balances the relationship between shallow image texture features and deep image semantic features by fusing multi-scale features and setting learnable weight coefficients, thereby the networks ability to extract pathological features at different scales is improved. At the same time,this method introduces the attention module and perform channel weighted summation on the feature maps output by the network to improve the feature expression ability of key feature channels, so that the network can pay more attention to the recognizable pathological feature areas in the image accurately. Finally, the multiple losses are used to further optimize the distribution of sample features in the feature space when the loss function is designed. As a result, the retrieval accuracy of 0.95 and 0.98 is achieved on the mAP@100 and mAP@20 indicators on the Mura dataset, which basically meets the retrieval accuracy requirements of the actual scene on the model.