Chinese Journal of Liquid Crystals and Displays, Volume. 38, Issue 3, 356(2023)
No-reference image quality assessment based on feature tokenizer and Transformer
Fig. 2. Tokenizer.（a）Filter-based tokenizer；（b）Recurrent tokenizer.
Fig. 3. Visualization of feature map.（a）Reference image；（b）Distorted image.（a1），（a2），（a3），（a4）and（b1），（b2），（b3），（b4）are feature maps extracted by the semantic feature extraction network for（a）and（b），respectively.
Fig. 4. Impact of the number of patches on performance.（a）PLCC；（b）SROCC.
Fig. 5. Number of parameters vs. performance on TID2013.（a）PLCC；（b）SROCC.
Table 1. Architecture of the semantic feature extraction module
View tableView in ArticleTable 1. Architecture of the semantic feature extraction module
Layer Block type Kernel size Channel Stride Num of blocks SE-1 Conv+BN+ReLU 3×3 16 1 ×1 SE-2 Conv+BN+ReLU 3×3 16 1 ×6 SE-3 Conv+BN+ReLU 3×3 32 2 ×1 Conv+BN+ReLU 3×3 32 1 ×5 SE-4 Conv+BN+ReLU 3×3 64 2 ×1 Conv+BN+ReLU 3×3 64 1 ×5
Table 2. Architecture of the low level feature extraction network
View tableView in ArticleTable 2. Architecture of the low level feature extraction network
Layer type Kernel size Channels Stride Padding conv 7×7 ［ 3 ，16 ］1×1 3 conv 5×5 ［16，6］ 1×1 2 conv 3×3 ［16，2］ 1×1 1 conv 3×3 ［32，32］ 1×1 1 conv 3×3 ［32，32］ 1×1 1
Table 3. Performance comparison on benchmark IQA datasets
View tableView in ArticleTable 3. Performance comparison on benchmark IQA datasets
NR-IQA methods LIVE CSIQ TID2013 LIVE-MD LIVE-CH PLCC SROCC PLCC SROCC PLCC SROCC PLCC SROCC PLCC SROCC Traditional BLIINDS-Ⅱ 0.920 0.919 0.534 0.570 0.628 0.536 0.845 0.827 0.450 0.405 DIIVINE 0.923 0.925 0.836 0.784 0.549 0.645 0.894 0.874 0.568 0.607 BRISQUE 0.942 0.939 0.829 0.850 0.651 0.573 0.921 0.897 0.585 0.607 NIQE 0.919 0.915 0.718 0.630 0.415 0.299 0.815 0.745 0.480 0.430 CORINA 0.943 0.942 0.781 0.714 0.613 0.549 0.915 0.900 0.662 0.618 FRIQUEE 0.962 0.948 0.863 0.839 0.704 0.669 0.940 0.925 0.720 0.720 CNN-based WaDiQaM 0.936 0.954 — — 0.787 0.761 — — 0.671 0.680 Rank 0.982 0.981 — — 0.799 0.780 — — — — DIQA 0.977 0.975 0.915 0.884 0.850 0.825 0.942 0.939 0.704 0.703 BIECON 0.960 0.958 0.823 0.815 0.762 0.717 0.933 0.909 0.613 0.595 BPSQM 0.963 0.973 0.915 0.874 0.885 0.862 — — — — CaHDC 0.964 0.965 0.914 0.903 0.878 0.862 0.950 0.927 0.744 0.738 AIGQA 0.957 0.960 0.952 0.927 0.893 0.871 0.947 0.933 0.761 0.751 ENOSS 0.961 0.966 0.959 0.954 0.891 0.874 0.914 0.885 — — Transformer based TRIQ 0.482 0.469 0.568 0.501 0.575 0.466 0.842 0.818 0.910 0.902 VTT-IQA 0.964 0.968 0.962 0.954 0.908 0.887 0.954 0.958 0.727 0.704
Table 4. Results on individual distortion types
View tableView in ArticleTable 4. Results on individual distortion types
Type JP2K JPEG WN GB FFR PLCC 0.970 0.966 0.988 0.970 0.938 SROCC 0.971 0.959 0.989 0.965 0.923
Table 5. Results on underwater IQA dataset
View tableView in ArticleTable 5. Results on underwater IQA dataset
Model SROCC KROCC BRISQUE 0.716 0.588 DIIVINE 0.611 0.481 FRIQUEE 0.626 0.490 UCIQE 0.518 0.386 UIQM 0.408 0.309 VTT-IQA 0.845 0.655
Table 6. Results of ablation experiments
View tableView in ArticleTable 6. Results of ablation experiments
Model PLCC SROCC VTT -IQA w/o LENet 0.951 0.938 VTT -IQA w/o STNet 0.887 0.871 VTT -IQA 0.962 0.954
Table 7. Performance comparison of different tokenizer strategies
View tableView in ArticleTable 7. Performance comparison of different tokenizer strategies
Tokenizer strategy PLCC SROCC Filter -based tokenizer 0.944 0.941 Recurrent tokenizer 0.964 0.968
Table 8. Impact of the number of feature tokens on model performance
View tableView in ArticleTable 8. Impact of the number of feature tokens on model performance
Number of vision tokens PLCC SROCC 8 0.944 0.940 16 0.964 0.968 24 0.962 0.962
Get Citation
Copy Citation Text
Wei SONG, Jia-jin LI, Xiao-chen LIU, Zhi-xiang LIU, Shao-hua SHI. No-reference image quality assessment based on feature tokenizer and Transformer[J]. Chinese Journal of Liquid Crystals and Displays, 2023, 38(3): 356
Category: Research Articles
Received: Jun. 29, 2022
Accepted: --
Published Online: Apr. 3, 2023
The Author Email: Wei SONG (wsong@shou.edu.cn)
