Global-local combined features to detect pain intensity from facial expression images with attention mechanism

Jiang Wu; Yi Shi; Shun Yan; Hong-Mei Yan

doi:10.1016/j.jnlest.2024.100260

Journal of Electronic Science and Technology, Volume. 22, Issue 3, 100260(2024)

Global-local combined features to detect pain intensity from facial expression images with attention mechanism

Jiang Wu1...2, Yi Shi2, Shun Yan3, and Hong-Mei Yan12,* |Show fewer author(s)

¹Yangtze Delta Region Institute (Huzhou), University of Electronic Science and Technology of China, Huzhou, 313001, China

²MOE Key Laboratory for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, 611731, China

³College of Engineering, University of California at Santa Barbara, Santa Barbara, 93106, USA

show less

Figures & Tables(10)

Fig. 1. Overall pre-processing pipeline: (a) original image, (b) facial localization, (c) facial alignment, and (d) facial cropping.

Download full size

View in Article

Fig. 2. Framework of the proposed GLA-CNN.

Download full size

View in Article

Fig. 3. Sample frames and their corresponding PSPI scores at the UNBC-McMaster Shoulder Pain database.

Download full size

View in Article

Fig. 4. Number of pictures per PSPI code class at the UNBC-McMaster Shoulder Pain database.

Download full size

View in Article

Fig. 5. Confusion matrix based on the proposed GLA-CNN model.

Download full size

View in Article

Fig. 6. Samples of attention map from “No Pain” to “Strong Pain” by different models at the UNBC-McMaster Shoulder Pain database.

Download full size

View in Article

Table 1. Number of image for four pain levels at the UNBC-McMaster Shoulder Pain database² according to the PSPI score.
View table
View in Article
Table 1. Number of image for four pain levels at the UNBC-McMaster Shoulder Pain database² according to the PSPI score.
PSPI score Pain level Number of images
0 No pain 2928
1 Weak pain 2909
2 Mild pain 2351
≥3 Strong pain 3102

Table 2. Comparison of the average performance of proposed GLA-CNN with VGG-16, separated local and global modules under 5-fold cross validation.

View table

View in Article

Table 2. Comparison of the average performance of proposed GLA-CNN with VGG-16, separated local and global modules under 5-fold cross validation.

Model	Accuracy	F1-score	Recall	Precision
Baseline(VGG-16)	51.72%	36.65%	34.55%	44.36%
LANet	55.53%	35.38%	33.78%	42.92%
GANet	55.51%	37.22%	33.91%	43.03%
GLA-CNN	56.45%	36.52%	34.08%	43.23%
GLA-CNN-BiLSTM	54.54%	33.15%	31.07%	42.02%

Table 3. Comparison of the proposed model with the state-of-the-art at the UNBC-McMaster Shoulder Pain database.

View table

View in Article

Table 3. Comparison of the proposed model with the state-of-the-art at the UNBC-McMaster Shoulder Pain database.

Ref.	Method	Level	Accuracy
[42]	Head analysis	4	28.10%
[43]	LBP	4	34.70%
[43]	LPQ	4	23.40%
[43]	BSIF	4	23.70%
[44]	CNN	4	35.30%
[16]	CNN	4	51.10%
[45]	MRN	4	46.18%
[46]	MA-NET	4	55.16%
[47]	Swin transformer	4	54.40%
[48]	POSTER V2	4	52.24%
This work	GLA-CNN	4	56.45%

Table 4. Comparison of different network performance with/without attentional mechanisms (GA and LA stand for global attention and local attention, respectively).

View table

View in Article

Table 4. Comparison of different network performance with/without attentional mechanisms (GA and LA stand for global attention and local attention, respectively).

Model	GA	LA	Accuracy	F1-score	Recall	Precision
VGG-16	×	×	51.72%	36.65%	34.55%	44.36%
GNet	×	×	53.42%	33.06%	30.18%	41.32%
GANet	√	×	55.51%	37.22%	33.91%	43.03%
LNet	×	×	53.28%	35.76%	33.07%	43.77%
LANet	×	√	55.53%	35.38%	33.78%	42.92%
GLA-CNN	√	√	56.45%	36.52%	34.08%	43.23%

Tools

Get Citation

Copy Citation Text

Jiang Wu, Yi Shi, Shun Yan, Hong-Mei Yan. Global-local combined features to detect pain intensity from facial expression images with attention mechanism[J]. Journal of Electronic Science and Technology, 2024, 22(3): 100260

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites

Paper Information

Category:

Received: Sep. 2, 2023

Accepted: May. 21, 2024

Published Online: Oct. 11, 2024

The Author Email: Yan Hong-Mei (hmyan@uestc.edu.cn)

DOI:10.1016/j.jnlest.2024.100260

Topics

laser devices and laser physics

Lasers and Laser Optics

Laser physics

laser manufacturing

Instrumentation, Measurement and Metrology

Table 1. Number of image for four pain levels at the UNBC-McMaster Shoulder Pain database2 according to the PSPI score.

Table 1. Number of image for four pain levels at the UNBC-McMaster Shoulder Pain database2 according to the PSPI score.

Table 2. Comparison of the average performance of proposed GLA-CNN with VGG-16, separated local and global modules under 5-fold cross validation.

Table 2. Comparison of the average performance of proposed GLA-CNN with VGG-16, separated local and global modules under 5-fold cross validation.

Table 3. Comparison of the proposed model with the state-of-the-art at the UNBC-McMaster Shoulder Pain database.

Table 3. Comparison of the proposed model with the state-of-the-art at the UNBC-McMaster Shoulder Pain database.

Table 4. Comparison of different network performance with/without attentional mechanisms (GA and LA stand for global attention and local attention, respectively).

Table 4. Comparison of different network performance with/without attentional mechanisms (GA and LA stand for global attention and local attention, respectively).

微信扫一扫：分享

Table 1. Number of image for four pain levels at the UNBC-McMaster Shoulder Pain database² according to the PSPI score.

Table 1. Number of image for four pain levels at the UNBC-McMaster Shoulder Pain database² according to the PSPI score.