An image quality assessment method based on multi-task self-supervised learning is proposed to address the existing deep learning-based image quality assessment methods, which suffer from overfitting and insufficient generalization performance due to insufficient labeled data. First,17 distortion type images are synthesized by the algorithm and the full reference mean deviation similarity index (MDSI) score and distortion type are used as 2 labels for the synthesized distortion images. Subsequently, multi-task self-supervised learning on vision transformer (ViT) for predicting MDSI scores and distortion types. Finally, the trained model is fine-tuned on the downstream task to migrate the semantic features learned from the upstream task to the downstream task. The method in this paper is fully compared with mainstream no reference image quality assessment (NR-IQA) methods on several publicly available image quality assessment datasets, and the test results on LIVE, CSIQ, TID2013, and CID2013 are all improved by 1 to 2 percentage points compared with the best performing algorithms, which indicates that the proposed algorithm outperforms most mainstream unreferenced image quality assessment algorithms.