Non-Contact Psychological Stress Detection Combining Heart Rate Variability and Facial Expressions

Lingqin Kong; Fei Chen; Yuejin Zhao; Liquan Dong; Ming Liu; Mei Hui

doi:10.3788/AOS202141.0310003

Acta Optica Sinica, Volume. 41, Issue 3, 0310003(2021)

Non-Contact Psychological Stress Detection Combining Heart Rate Variability and Facial Expressions

Lingqin Kong, Fei Chen, Yuejin Zhao^*, Liquan Dong, Ming Liu, and Mei Hui

Beijing Key Laboratory for Precision Optoelectronic Measurement Instrument and Technology, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China

show less

Abstract Get PDF(in Chinese)

Figures & Tables(12)

Fig. 1. Process of the establishing a psychological stress assessment model

Download full size

Fig. 2. Extraction flow chart of the non-contact HRV

Download full size

Fig. 3. Signal diagram during HRV extraction. (a) Mean gray value; (b) pulse wave signal after processing; (c) timing diagram of R-R interval

Download full size

Fig. 4. Diagram of the experimental setup

Download full size

Fig. 5. Box plot of the HRV feature distribution. (a) Mean HR; (b) SDNN; (c) PNN₅₀; (d) RMSSD; (e) N_LF; (f) N_HF; (g) X_LF/X_HF; (h) X_SD2; (i) D₂

Download full size

Fig. 6. Box plot of the expression pressure value

Download full size

Table 1. Time domain analysis indicators of the HRV

View table

Table 1. Time domain analysis indicators of the HRV

Index	Definition	Unit	Meaning
Mean HR	the average number ofheartbeats per minute	BPM	reflect the heart beat rate
PNN₅₀	the heart rate of adjacent R-R intervalsgreater than 50ms as a percentageof all NN intervals	%	reflect the activity ofparasympathetic nerve
SDNN	$\sqrt[]{\overset{N}{\sum_{i = 1}} (X_{i}^{RR} - X^{\bar{RR}})^{2} / (N - 1)}$	ms	reflect the level that the heart ratevariability deviates from the average
RMSSD	$\sqrt[]{\frac{\overset{N - 1}{\sum_{i = 1}} (X_{i + 1}^{RR} - X_{i}^{RR})^{2}}{N - 1}}$	ms	measure the regulation effect ofparasympathetic nerve on heart rate

Table 2. Frequency domain analysis indicators of the HRV

View table

Table 2. Frequency domain analysis indicators of the HRV

Index	Definition	Unit	Meaning
X_VLF	ultra low frequency power (≤0.04 HZ)	ms²	related to body adjustment
X_LF	low frequency power(0.04--0.15 HZ)	ms²	reflect the activity of sympathetic nerve
X_HF	high frequency power(0.15--0.4 HZ)	ms²	reflect the activity of parasympathetic nerve
N_LF	N_LF= $\frac{X_{LF}}{X_{TP} - X_{VLF}}$ ×100%	%	standardized low frequency power,direct response to the activity ofsympathetic nerve.TP:total power
N_HF	N_HF= $\frac{X_{HF}}{X_{TP} - X_{VLF}}$ ×100%	%	standardized high frequency power,direct response to the activity ofparasympathetic nerve.TP:total power
X_LF/X_HF	power ratio of low frequencyband to high frequency band	-	reflect the balance of sympatheticand parasympathetic nerves
P_VLF	peak point of ultra low frequency power	Hz	-
P_LF	peak point of low frequency power	Hz	-
P_HF	peak point of high frequency power	Hz	-

Table 3. Non-linear analysis indicators of the HRV

View table

Table 3. Non-linear analysis indicators of the HRV

Index	Definition	Unit	Meaning
X_SD1	the standard deviation of thevertical line in the Poincaré chart	ms	related to sympathetic nerve activity
X_SD2	the standard deviation along themarked line in the Poincaré chart	ms	related to parasympathetic nerve activity
D₂	correlation dimension	-	reflect the heart's ability toadapt to the environment
α₁	short-term volatility slope in trend volatility analysis	-	-
α₂	long-term volatility slope in trend volatility analysis	-	-

Table 4. Results of the HRV

View table

Table 4. Results of the HRV

Name	Normal	Stress
Mean HR /BPM	78.231	84.421
PNN₅₀ /%	33.882	29.164
SDNN /ms	52.034	46.424
RMSSD /ms	38.623	33.862
X_VLF /ms²	52.317	42.347
X_LF /ms²	375.427	327.135
X_HF /ms²	961.326	613.463
N_LF /%	28.085	34.779
N_HF /%	71.915	65.221
X_LF/X_HF	0.417	0.541
P_VLF /Hz	0.039	0.039
P_LF /Hz	0.110	0.117
P_HF /Hz	0.285	0.297
X_SD1 /ms	38.457	37.837
X_SD2 /ms	41.626	31.976
D₂	3.541	2.036
α₁	0.668	0.636
α₂	0.220	0.236

Table 5. Results of training using HRV and facial expressions
View table
Table 5. Results of training using HRV and facial expressions
Model Kernel type Gamma C Accuracy /%
SVM Gaussiankernel function 0.002 6.679 81.4

Table 6. Results of the comparative experimentsunit: %
View table
Table 6. Results of the comparative experimentsunit: %
Method Accuracy
HRV feature in Ref. [10] 65.1
HRV feature in ours 73.9
Expression 71.9
HRV & Expression 81.4

Tools

Get Citation

Copy Citation Text

Lingqin Kong, Fei Chen, Yuejin Zhao, Liquan Dong, Ming Liu, Mei Hui. Non-Contact Psychological Stress Detection Combining Heart Rate Variability and Facial Expressions[J]. Acta Optica Sinica, 2021, 41(3): 0310003

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites