The problems of breast tumor segmentation from ultrasound images, such as low contrast between the tumor and the normal tissue, blurred boundaries, complex shapes and positions of tumors, and high noise in images, are a concern for researchers. This paper presents a hierarchical transformer with a multiscale parallel aggregation network for breast tumor segmentation. The encoder uses MiT-B2 to establish long-range dependencies and effectively extract features at different resolutions. At the skip connection between the encoder and the decoder, a cascaded module incorporating a multi-scale receptive field block and shuffle attention (SA) mechanism is constructed. receptive field block is used to capture multi-scale local information of the tumor, addressing the problem of high similarity between the lesion and surrounding normal tissue. The SA mechanism accurately identifies and localizes tumors while suppressing noise interference. In the decoder, an aggregation module is constructed to progressively fuse features from parallel branches to enhance segmentation accuracy. The experimental results on the BUSI dataset show that, compared to TransFuse, the proposed model achieves improvements of 3.21% and 3.19% in the Dice and intersection over union metrics, respectively. The model also shows excellent results for the other two datasets.