Multi-Angle Facial Expression Recognition Algorithm Combined with Dual-Channel WGAN-GP

Fig. 8. Confusion matrix of expression recognition under different angles on Multi-PIE dataset. (a) 0° rotation; (b) 30° rotation; (c) 60° rotation; (d) 90° rotation

Download full size

Fig. 9. Partial error samples on Multi-PIE dataset

Download full size

Fig. 10. Comparison of network loss function changes. (a) TP-GAN discriminant loss; (b) TP-GAN adversarial loss; (c) proposed model discriminant loss; (d) proposed model adversarial loss

Download full size

Fig. 11. Comparison of facial expression images generation effect after model ablation. (a) Profile; (b) without local path; (c) without Wasserstein; (d) without GP; (e) proposed method; (f) real frontal

Download full size

Fig. 12. Comparison of facial expression recognition rate under different ablation results

Download full size

Fig. 13. Comparison of frontalization generation effect. (a) Real profile; (b) proposed method; (c) TP-GAN^[11]; (d) CAPG-GAN^[9]; (e) FI-GAN^[23]; (f) MTDNN^[24]; (g) 3DMM^[25]; (h) real frontal

Download full size

Table 1. Structure of improved MobileNetV3

View table

Table 1. Structure of improved MobileNetV3

Input	Operator	Channel	SE	Stride	Repeat
128²×3	Conv 3×3	16		2
64²×16	Bottleneck 3×3	16	√	2
32²×16	Bottleneck 3×3	24		2
16²×24	Bottleneck 5×5	40	√	2
8²×40	Bottleneck 5×5	40	√	1	×2
8²×40	Bottleneck 5×5	48	√	1	×2
8²×48	Bottleneck 5×5	96	√	2
4²×96	Bottleneck 3×3	96	√	1	×2
4²×96	Conv 1×1	256	√	1
4²×256	Pooling 4×4			1
1²×256	Conv 1×1	512		1
1²×512	Conv 1×1	7		1

Table 2. Multi-angle facial expression recognition rate on KDEF dataset
View table
Table 2. Multi-angle facial expression recognition rate on KDEF dataset
Angle /° -90 -45 0 45 90
Accuracy /% 83.88 87.76 93.27 88.37 82.65

Table 3. Multi-angle facial expression recognition rate on Multi-PIE dataset
View table
Table 3. Multi-angle facial expression recognition rate on Multi-PIE dataset
Angle /° -90 -60 -30 0 30 60 90
Accuracy /% 81.83 86.00 88.67 92.17 89.17 85.50 81.33

Table 4. Expression recognition rate under different loss functions on Multi-PIE dataset
View table
Table 4. Expression recognition rate under different loss functions on Multi-PIE dataset
Method 30° 60° 90°
Without L_sym （k₃=0） 87.00 81.83 76.17
Without L_ip （k₄=0） 82.67 73.33 62.50
Proposed method 89.17 85.50 81.33

Table 5. Recognition results of frontal expressions under different models
View table
Table 5. Recognition results of frontal expressions under different models
Method Accuracy /% Parameter computation /MB
VGG-19^［20］ 82.15 70.45
ResNet-50^［21］ 84.94 23.60
Xception^［22］ 90.55 20.88
MobileNetV1^［14］ 90.24 3.24
Proposed method 92.17 0.95

Table 6. Comparison of multi-angle facial expression recognition rate with existing methods on KDEF dataset
View table
Table 6. Comparison of multi-angle facial expression recognition rate with existing methods on KDEF dataset
Method -90˚ -45˚ 0˚ 45˚ 90˚
Reference［5］ 74.50 72.00 70.50 77.40 79.50
Reference［6］ 80.39 84.07 77.03
Reference［7］ 86.67 83.81 76.67
Proposed method 83.88 87.76 93.27 88.37 82.65

Table 7. Comparison of multi-angle facial expression recognition rate with existing methods on Multi-PIE dataset

View table

Table 7. Comparison of multi-angle facial expression recognition rate with existing methods on Multi-PIE dataset

Method	-90˚	-60˚	-30˚	0˚	30˚	60˚	90˚
Reference［26］			93.10	92.80	92.20
Reference［27］			92.58	89.65	94.51
Reference［28］	79.00	83.00	80.00	86.00	83.50	82.00	76.00
Reference［29］				88.20	87.30	83.80	78.80
Reference［30］				83.30	85.80	81.60	92.50
Proposed method	81.83	86.00	88.67	92.17	89.17	85.50	81.33

Tools

Get Citation

Copy Citation Text

Yuan Deng, Yiping Shi, Jie Liu, Yueying Jiang, Yamei Zhu, Jin Liu. Multi-Angle Facial Expression Recognition Algorithm Combined with Dual-Channel WGAN-GP[J]. Laser & Optoelectronics Progress, 2022, 59(18): 1810013

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites

Paper Information

Category: Image Processing

Received: Jun. 9, 2021

Accepted: Aug. 10, 2021

Published Online: Aug. 22, 2022

The Author Email: Shi Yiping (810992866@qq.com)

DOI:10.3788/LOP202259.1810013

Topics

laser devices and laser physics

Lasers and Laser Optics

Laser physics

laser manufacturing

Instrumentation, Measurement and Metrology

Table 1. Structure of improved MobileNetV3

Table 1. Structure of improved MobileNetV3

Table 2. Multi-angle facial expression recognition rate on KDEF dataset

Table 2. Multi-angle facial expression recognition rate on KDEF dataset

Table 3. Multi-angle facial expression recognition rate on Multi-PIE dataset

Table 3. Multi-angle facial expression recognition rate on Multi-PIE dataset

Table 4. Expression recognition rate under different loss functions on Multi-PIE dataset

Table 4. Expression recognition rate under different loss functions on Multi-PIE dataset

Table 5. Recognition results of frontal expressions under different models

Table 5. Recognition results of frontal expressions under different models

Table 6. Comparison of multi-angle facial expression recognition rate with existing methods on KDEF dataset

Table 6. Comparison of multi-angle facial expression recognition rate with existing methods on KDEF dataset

Table 7. Comparison of multi-angle facial expression recognition rate with existing methods on Multi-PIE dataset

Table 7. Comparison of multi-angle facial expression recognition rate with existing methods on Multi-PIE dataset