Opto-Electronic Engineering, Volume. 52, Issue 1, 240234(2025)
Lightweight Swin Transformer combined with multi-scale feature fusion for face expression recognition
Figures & Tables(20)
Fig. 3. Self-attention computing area. (a) MSA; (b) W-MSA; (c) SW-MSA
Fig. 6. A visual view of the BN, LN, and BCN standardization technology
Fig. 8. Activation maps of the model before and after adding EMA module
Fig. 10. Confusion matrix validation results on JAFFE. (a) Original Swin Transformer model; (b) Improved Swin Transformer model
Fig. 11. Confusion matrix validation results on RAF-DB. (a) Original Swin Transformer model; (b) Improved Swin Transformer model
Fig. 12. Confusion matrix validation results on FERPLUS. (a) Original Swin Transformer model; (b) Improved Swin Transformer model
Fig. 13. Confusion matrix validation results on FANE. (a) Original Swin Transformer model; (b) Improved Swin Transformer model
Table 1. Comparison of parameters before and after the model is improved
View tableView in ArticleTable 1. Comparison of parameters before and after the model is improved
Model EMA module SPST module Parameters Original Swin Transformer × × 27,524,737 Improved Swin Transformer √ × 27,526,225 Improved Swin Transformer × √ 23,185,251 Improved Swin Transformer √ √ 23,186,739
Table 2. Experimental comparison of replacing SPST modules in different stages
View tableView in ArticleTable 2. Experimental comparison of replacing SPST modules in different stages
Position Swin Transformer block SPST block Parameters RACC/% GFLOPs/G FPS Stage1 √ 34,331,981 72.33 19.06 86 Stage2 √ 29,625,428 75.27 12.44 152 Stage3 √ 24,190,413 82.17 5.84 281 Stage4 √ 23,185,251 86.86 4.12 335 Stage4 √ 27,524,737 85.69 4.51 301
Table 3. Entropy comparison of activation maps
View tableView in ArticleTable 3. Entropy comparison of activation maps
Model Anger Disgust Fear Happy Sad Surprise Original Swin Transformer 10.5974 10.5325 10.4282 10.6150 10.5980 10.6626 Improved Swin Transformer 8.2437 9.4190 9.2204 8.1102 8.9906 8.9113
Table 4. Configuration of the experimental environment
View tableView in ArticleTable 4. Configuration of the experimental environment
Configuration name Environmental parameter CPU Inter (R) Core (TM) i5-12400F 2.50 GHz GPU NVIDIA GeForce RTX 3060 (12 GB) Memory 16 G Python 3.9.19 CUDA 11.8 Torch 2.0.0
Table 5. Accuracy of embedding the EMA module behind different stages
View tableView in ArticleTable 5. Accuracy of embedding the EMA module behind different stages
Position RACC/% Parameters JAFFE FERPLUS RAF-DB FANE After stage1 95.57 85.53 86.80 68.84 23,185,635 After stage2 97.56 86.46 87.29 70.11 23,186,739 After stage3 96.80 85.56 86.99 68.60 23,191,107 After stage4 95.87 85.76 86.67 69.37 23,187,875
Table 6. Results of ablation experiments on FERPLUS, RAF-DB, and FANE
View tableView in ArticleTable 6. Results of ablation experiments on FERPLUS, RAF-DB, and FANE
SPST module EMA module RACC/% Parameters GFLOPs/G FPS FERPLUS RAF-DB FANE × × 85.43 85.69 68.47 27,524,737 4.51 301 × √ 85.73 86.99 69.67 27,526,225 4.52 297 √ × 85.87 86.86 69.72 23,185,251 4.12 335 √ √ 86.46 87.29 70.11 23,186,739 4.13 330
Table 7. Accuracy comparsion of different networks on JAFFE，FERPLUS, and RAF-DB
View tableView in ArticleTable 7. Accuracy comparsion of different networks on JAFFE，FERPLUS, and RAF-DB
Model ACC/% JAFFE FERPLUS RAF-DB ARBEx[ 9 ]96.67 —— —— LBP+HOG[ 7 ]96.05 —— —— SCN[ 4 ]86.33 85.97 87.03 RAN[ 8 ]88.67 83.63 86.90 EfficientNetB0[ 25 ]—— 85.01 84.21 MobileNetV2[ 26 ]—— 84.03 83.54 MobileNetV3[ 27 ]—— 84.97 84.88 Ad-Corre[ 28 ]—— —— 86.96 POSTER[ 19 ]—— —— 86.03 R3HO-Net[ 29 ]—— —— 85.52 Ada-CM[ 30 ]—— —— 84.13 Swin Transformer (base) 95.12 85.43 85.69 Ours 97.56 86.46 87.29
Tools
Get Citation
Copy Citation Text
Yanqiu Li, Shengzhao Li, Guangling Sun, Pu Yan. Lightweight Swin Transformer combined with multi-scale feature fusion for face expression recognition[J]. Opto-Electronic Engineering, 2025, 52(1): 240234
Paper Information
Category: Article
Received: Oct. 7, 2024
Accepted: Dec. 3, 2024
Published Online: Feb. 21, 2025
The Author Email: Sun Guangling (孙光灵)
© Copyright 2018-2021 | Chinese Laser Press.
All Rights Reserved 沪ICP备15018463号-20