MPED: A Multi-Modal Physiological Emotion Database for Discrete Emotion Recognition

Tengfei Song, Wenming Zheng, Cheng Lu, Yuan Zong, Xilei Zhang, Zhen CuiIEEE Access 2019 · DOI

Introduction

Traditional emotion recognition systems often rely on external cues like facial expressions or vocal tone. However, these "non-physiological" signals can be easily manipulated or suppressed, leading to inaccuracies in detecting true internal states. To address this, researchers have developed the Multi-Modal Physiological Emotion Database (MPED), focusing on intrinsic mental states reflected through brainwaves and autonomic nervous system activity.

The MPED Dataset

The MPED is a standardized database designed to facilitate discrete emotion recognition. It includes data from 23 healthy Chinese volunteers who were exposed to 28 standardized video clips. These clips, selected from a massive pool of over 1,500, cover seven distinct emotional categories: joy, funny, anger, fear, disgust, sadness, and neutrality.

Sensor placement for multi-modal data collection: (a) 62-channel EEG, (b) GSR on fingers, (c) RSP belt, and (d) ECG on limbs.

Unlike existing datasets like DEAP or SEED, MPED is specifically optimized for discrete emotion classification rather than dimensional scales (valence/arousal). The recorded signals include:

  • 62-channel EEG: Capturing high-resolution brain activity.
  • Galvanic Skin Response (GSR): Measuring skin conductance related to arousal.
  • Respiration (RSP): Tracking breathing patterns.
  • Electrocardiogram (ECG): Monitoring heart rate variability.

Methodology: The Attention-LSTM Model

To process this complex, sequential data, the researchers introduced the Attention-Long Short-Term Memory (A-LSTM) architecture. While standard LSTMs are effective for time-series data, they often fail to distinguish between noise and relevant emotional markers.

The A-LSTM addresses this by incorporating three specialized attention branches that create "masks" for:

  1. Input Data: Filtering out noise in the raw physiological signals.
  2. Hidden States: Prioritizing the most relevant past information.
  3. Memory Cells: Determining which long-term features are critical for classification.

This mechanism allows the model to dynamically adapt to the high degree of individual variability inherent in physiological responses.

Key Findings and Results

The study yielded several significant insights into the physiological nature of emotion:

  • EEG Dominance: EEG signals proved to be the most effective modality for distinguishing between emotions, significantly outperforming GSR, RSP, and ECG.
  • Neural Correlates: The researchers confirmed that energy in the prefrontal cortex—specifically in the beta and gamma frequency bands—is a strong indicator of emotional valence.
  • Emotion Distinctiveness: "Funny" was found to be more easily recognized than "joy," likely due to the higher intensity of physiological arousal associated with humor. Similarly, "fear" showed unique neural signatures that made it more distinguishable from other negative emotions like anger or disgust.

In performance benchmarks, the A-LSTM model achieved an accuracy of approximately 42% for seven-class emotion recognition using EEG signals. This outperformed standard LSTMs (~38%) and traditional Support Vector Machines (~31%), demonstrating the value of the attention mechanism.

Conclusion and Future Directions

The MPED provides a robust foundation for moving emotion recognition toward more "honest" physiological signals. While the A-LSTM model shows promise in handling individual differences, future work will focus on adaptive fusion techniques to better weight different modalities and reducing the time window required for real-time applications.

Summary updated

BibTeX
@article{song2019mped,
  author = {Tengfei Song and Wenming Zheng and Cheng Lu and Yuan Zong and Xilei Zhang and Zhen Cui},
  title = {{MPED: A Multi-Modal Physiological Emotion Database for Discrete Emotion Recognition}},
  journal = {IEEE Access},
  year = {2019},
  doi = {10.1109/access.2019.2891579},
}
Figure