A novel efficient method based on the ultrasound radio frequency (RF) signals is proposed to distinguish the breast tumors grades. First, we utilize the multi-scale geometric characteristic of Shearlet transformation to extract the multi-scale and multi-directional features of ultrasound RF signal, and then reduce the high-dimensional Shearlet features by multi-scale directional binary pattern which can effectively preserve the sufficient discriminated information. At last, we draw on the feature difference between different grades of breast tumors to design a cascade binary tree SVM classifier which not only overcome the problem of sample quantity disequilibrium but also conform to the subjective diagnosis rule of sonographer. Extensive experiments on 928 breast ultrasound RF signals collected from the hospital demonstrate the effectiveness of the new proposed method and its precision, sensitivity, specificity, PPV, NPV and MCC are 89.29%, 75.62%, 94.54%, 97%, 98.3% and 81.01%, respectively.