Personnel emotion recognition model for Internet of vehicles security monitoring in community public space

Erkang Fu, Xi Li, Yao Zhi, Yuxin Ren, Yuanhao Wu, Qiqi Fan

Abstract

In the Internet of Vehicles (IoV), real-time emotion recognition is vital for preventing fatigue-related accidents and ensuring pedestrian safety. This study proposes a facial emotion recognition model specifically designed for the variable conditions of IoV environments. By utilizing Galvanic Skin Response (GSR) as an objective physiological ground truth, the model overcomes the inaccuracies of subjective labeling. The proposed architecture, combining a convolutional codec with a Support Vector Machine (SVM) classifier, achieved a matching accuracy of 82.01% on natural scene video data, significantly outperforming standard deep learning models like ResNet18.

Introduction

The rapid advancement of the Internet of Vehicles (IoV) has turned intelligent networked automobiles into critical terminal nodes. A key challenge in this domain is the real-time monitoring of driver and pedestrian emotional states to prevent fatigue driving and malicious collisions. However, traditional emotion recognition models often fail in real-world IoV scenarios due to environmental noise—such as varying camera angles and poor lighting—and the inherent subjectivity of self-reported emotional data.

Methodology

To address these challenges, this research introduces a robust framework that integrates physiological signals with computer vision.

Objective Labeling via GSR

Instead of relying on subjective annotations, the study employs Galvanic Skin Response (GSR) to define emotional ground truths. GSR measures skin conductivity changes driven by the autonomic nervous system, providing an honest reflection of emotional arousal rather than potentially deceptive facial expressions.

The relationship between common emotions and GSR signals.

Data Preprocessing and Filtering

Raw physiological data often contains noise that can obscure emotional markers. The researchers applied a Savitzky–Golay filter to smooth the GSR signals, facilitating the categorization of video segments into "Happy," "Quiet," or "Unhappy" states based on physiological trends.

The principle of Savitzky–Golay filtering for smoothing physiological data.

Model Architecture

The core of the system is a hybrid Codec-SVM model. A convolutional codec extracts abstract features from facial images, effectively filtering out "style" noise like lighting fluctuations and background variations. These features are then classified using a Support Vector Machine (SVM) with a Gaussian kernel. A temporal logic algorithm aggregates frame-level predictions into a final label for each 15-second video clip, accounting for the persistence of human emotion.

Results and Discussion

The model was tested on a custom dataset of 189 videos captured in natural scenes mimicking IoV environments. The proposed system achieved an 82.01% matching accuracy with GSR labels, correctly identifying 155 videos. In comparison, standard architectures like ResNet18 and VGG16 achieved only approximately 69.84% accuracy on the same dataset.

The results indicate that the "Quiet" (neutral) state is the most prevalent in natural scenes. The model's success is attributed to its ability to ignore irrelevant background data through feature abstraction and its reliance on physiological ground truths.

Conclusion

This research demonstrates a high-accuracy emotion recognition model tailored for IoV security. By bridging the gap between computer vision and physiological monitoring, the system provides a more reliable means of detecting emotional states in real-world conditions. Future work will focus on improving recognition for negative emotions and handling extreme head poses and occlusions common in driving scenarios.

Summary updated

Abstract

Abstract In recent years, the Internet of vehicles (IOV) with intelligent networked automobiles as terminal node has gradually become the development trend of automotive industry and research hot spot in related fields. This is due to its characteristics of intelligence, networking, low-carbon and energy saving. Real time emotion recognition for drivers and pedestrians in the community can be utilized to prevent fatigue driving and malicious collision, keep safety verification and pedestrian safety detection. This paper mainly studies the face emotion recognition model that can be utilized for IOV. Considering the fluctuation of image acquisition perspective and image quality in the application scene of IOV, the natural scene video similar to vehicle environment and its galvanic skin response (GSR) are utilized to make the testing set of emotion recognition. Then an expression recognition model combining codec and Support Vector Machine classifier is proposed. Finally, emotion recognition testing is completed on the basis of Algorithm 1. The matching accuracy between the emotion recognition model and GSR is 82.01%. In the process of model testing, 189 effective videos are involved and 155 are correctly identified.
BibTeX
@inproceedings{fu2021personnel,
  author = {Erkang Fu and Xi Li and Yao Zhi and Yuxin Ren and Yuanhao Wu and Qiqi Fan},
  title = {{Personnel emotion recognition model for Internet of vehicles security monitoring in community public space}},
  booktitle = {EURASIP},
  year = {2021},
  doi = {10.1186/s13634-021-00789-5},
}
Figure