Laser & Optoelectronics Progress, Volume. 58, Issue 6, 610005(2021)

Facial Expression Recognition by Merging Multilayer Features of Lightweight Convolutional Networks

Shen Hao1,2,3, Meng Qinghao1,2,3, and Liu Yinbo1,2,3、*
Author Affiliations
  • 1School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China
  • 2Institute of Robotics and Autonomous Systems, Tianjin University, Tianjin 300072, China
  • 3Tianjin Key Laboratory of Process Detection and Control, Tianjin 300072, China
  • show less
    References(27)

    [1] Mehrabian A, Russell J A[M]. An approach to environmental psychology, 56-63(1974).

    [2] Huang Y X, Chen F, Lü S et al. Facial expression recognition: a survey[J]. Symmetry, 11, 1189(2019).

    [3] Li S, Deng W H. Deep facial expression recognition: a survey[J]. IEEE Transactions on Affective Computing(2020).

    [4] Ye J H, Zhu J T, Jiang A W et al. Facial expression recognition: a survey[J]. Journal of Data Acquisition and Processing, 35, 21-34(2020).

    [5] Shan C F, Gong S G. McOwan P W. Facial expression recognition based on local binary patterns: a comprehensive study[J]. Image and Vision Computing, 27, 803-816(2009).

    [6] Dalal N, Triggs B. Histograms of oriented gradients for human detection[C]. //2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), June 20-25, 2005, San Diego, CA, USA., 886-893(2005).

    [7] Bashyal S, Venayagamoorthy G K. Recognition of facial expressions using Gabor wavelets and learning vector quantization[J]. Engineering Applications of Artificial Intelligence, 21, 1056-1064(2008).

    [8] Dino H I, Abdulrazzaq M B. Facial expression classification based on SVM, KNN and MLP classifiers[C]. //2019 International Conference on Advanced Science and Engineering (ICOASE), April 2-4, 2019, Zakho-Duhok, Iraq., 70-75(2019).

    [9] Zhong W, Huang Y L. Afacial expression recognition algorithm based on feature fusion and hierarchical decision tree technology[J]. Computer Engineering & Science, 39, 393-398(2017).

    [10] Wang Y B, Ai H Z, Wu B et al. Real time facial expression recognition with AdaBoost[C]. //Proceedings of the 17th International Conference on Pattern Recognition, ICPR 2004, August 26-26, 2004, Cambridge, UK., 926-929(2004).

    [11] Yu Z D, Zhang C. Image based static facial expression recognition with multiple deep network learning. [C]∥ICMI'15: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, November 9-13, 2015, Seattle, WA, USA. Seattle: ACM, 435-442(2015).

    [12] Zhou T, Lü X Q, Ren G Y et al. Facial expression classification based on ensemble convolutional neural network[J]. Laser & Optoelectronics Progress, 57, 141501(2020).

    [13] Li H, Li G M. Research on facial expression recognition based on LBP and DeepLearning[C]. //2019 International Conference on Robots & Intelligent System (ICRIS), June 15-16, 2019, Haikou, China., 94-97(2019).

    [14] Mollahosseini A, Chan D, Mahoor M H. Going deeper in facial expression recognition using deep neural networks[C]. //2016 IEEE Winter Conference on Applications of Computer Vision (WACV), March 7-10, 2016, Lake Placid, NY, USA., 1-10(2016).

    [15] Zhang H Y, Wang H S. Double-channel facial expression recognition based on local binary pattern and gradient features[J]. Laser & Optoelectronics Progress, 57, 141005(2020).

    [16] Arriaga O, Valdenegro-Toro M, Plöger P. Real-time convolutional neural networks for emotion and gender classification[EB/OL]. (2017-10-20)[2020-06-29].https:∥arxiv.org/abs/1710.07557.

    [17] Lü H, Tong Q Q, Yuan Z Y. Realtime architecture for facial expression recognition in complex scenes based on face region segmentation[J]. Computer Engineering and Applications, 56, 134-140(2020).

    [18] Li Y, Lin X Z, Jiang M Y. Facial expression recognition with cross-connect LeNet-5 network[J]. Acta Automatica Sinica, 44, 176-182(2018).

    [19] He Z C, Zhao L Z, Chen C. Convolution neural network with multi-resolution feature fusion for facial expression recognition[J]. Laser & Optoelectronics Progress, 55, 071503(2018).

    [20] Sandler M, Howard A, Zhu M L et al. MobileNetV2 inverted residuals and linear bottlenecks[C]. //2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 18-23, 2018, Salt Lake City, UT, USA, 4510-4520(2018).

    [21] Misra D. Mish:a self regularized non-monotonic neural activation function[EB/OL]. (2019-08-13)[2020-06-29]. https://arxiv.org/abs/1908.08681?context=cs.LG.

    [22] Li S, Deng W H, Du J P. Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild[C]. //2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 21-26, 2017, Honolulu, HI, USA., 2584-2593(2017).

    [23] Mollahosseini A, Hasani B, Mahoor M H. AffectNet: a database for facial expression, valence, and arousal computing in the wild[J]. IEEE Transactions on Affective Computing, 10, 18-31(2019).

    [24] Liu Z W, Li S, Deng W H. Boosting-POOF: boosting part based one vs one feature for facial expression recognition in the wild[C]. //2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), May 30 - June 3, 2017, Washington, DC, USA, 967-972(2017).

    [25] Simonyan K. Zisserman A. Verydeep convolutional networks for large-scale image recognition[EB/OL]. (2015-04-10)[ 2020-06-29]. https://arxiv.org/abs/1409.1556.

    [26] Li Y, Zeng J B, Shan S G et al. Occlusion aware facial expression recognition using CNN with attention mechanism[J]. IEEE Transactions on Image Processing, 28, 2439-2450(2019).

    [27] Cao S, Yao Y Q, An G Y. E2-capsule neural networks for facial expression recognition using AU-aware attention[J]. IET Image Processing, 14, 2417-2424(2020).

    Tools

    Get Citation

    Copy Citation Text

    Shen Hao, Meng Qinghao, Liu Yinbo. Facial Expression Recognition by Merging Multilayer Features of Lightweight Convolutional Networks[J]. Laser & Optoelectronics Progress, 2021, 58(6): 610005

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Image Processing

    Received: Jun. 30, 2020

    Accepted: --

    Published Online: Mar. 11, 2021

    The Author Email: Yinbo Liu (liuyinbo@tju.edu.cn)

    DOI:10.3788/LOP202158.0610005

    Topics