Datasets collected and annotated manually are inevitably contaminated with label noise, which negatively affects the generalization ability of image classification models. Therefore, designing robust classification algorithms for datasets with label noise has become a hot research topic. The main issue with existing methods is that self-supervised learning pre-training is time-consuming and still includes a large number of noisy samples after sample selection. This paper introduces the AllMix model, which reduces the time required for pre-training. Based on the DivideMix model, the AllMatch training strategy replaces the original MixMatch training strategy. The AllMatch training strategy uses focal loss and generalized cross-entropy loss to optimize the loss calculation for labeled samples. Additionally, it introduces a high-confidence sample semi-supervised learning module and a contrastive learning module to fully learn from unlabeled samples. Experimental results show that on the CIFAR10 dataset, the existing pre-trained label noise classification algorithms are 0.7%, 0.7%, and 5.0% higher in performance than those without pre-training for 50%, 80%, and 90% symmetric noise ratios, respectively. On the CIFAR100 dataset with 80% and 90% symmetric noise ratios, the model performance is 2.8% and 10.1% higher, respectively.