Abstract
Evaluation of quality of experience (QoE) based on electroencephalography (EEG) has received great attention due to its capability of real-time QoE monitoring of users. However, it still suffers from rather low recognition accuracy. In this paper, we propose a novel method using deep neural networks toward improved modeling of EEG and thereby improved recognition accuracy. In particular, we aim to model spatio-temporal characteristics relevant for QoE analysis within learning models. The results demonstrate the effectiveness of the proposed method.
Introduction
Understanding "Quality of Experience" (QoE)—the degree of delight or annoyance a user feels while consuming multimedia content—is essential for personalized service delivery. Traditional methods rely on explicit user feedback, such as surveys, which are subjective and cannot provide real-time insights. Implicit monitoring via electroencephalography (EEG) offers a promising alternative for real-time QoE assessment. However, EEG data is notoriously noisy, and previous recognition models often failed to exceed 70-80% accuracy. A primary limitation of these models was treating EEG electrodes as independent data points, neglecting the critical spatial-temporal relationships between different brain regions.
Methodology
To overcome these limitations, this research proposes a deep learning framework based on Convolutional Neural Networks (CNNs). While CNNs are typically used for image processing, they are adapted here to capture the spatial relationships of brain activity by transforming 1D EEG signals into 2D matrices.
Spatial and Relational Features
Rather than relying solely on local Power Spectral Density (PSD), the model utilizes relational features to capture how different brain regions interact. These include the Pearson Correlation Coefficient (PCC) for linear relationships, Phase Locking Value (PLV) for phase synchronization, and Transfer Entropy (TE) for directional information flow. To ensure these relationships are meaningful to the CNN, electrodes are arranged in the input matrix based on their physical distance on the scalp, allowing the network to learn localized patterns of neural activity.
Multi-Band Input
The model processes EEG data across 10 distinct frequency bands (ranging from Delta to Gamma). These bands are treated as separate input channels, analogous to the RGB channels in a standard image, providing the network with a rich, multi-dimensional view of brain state.
Experimental Results
The proposed method was evaluated using the DEAP database, a benchmark dataset containing 32-channel EEG recordings from subjects watching music videos. The task was defined as a binary classification of "liking" versus "disliking."
- Peak Performance: The combination of the PLV relational feature and a medium-depth CNN architecture (CNN2) achieved a record-breaking F1-score of 0.969. This significantly outperforms previous state-of-the-art benchmarks on the same dataset, which typically reached approximately 0.880.
- Regression Accuracy: For predicting exact preference scores on a 1–9 scale, the model achieved a Root Mean Square Error (RMSE) of 1.228.
- Connectivity vs. Locality: Relational features consistently outperformed traditional PSD features, confirming that user preference is a networked phenomenon emerging from interactions between brain regions rather than isolated activity.
Discussion and Limitations
While the model shows high overall accuracy, a "negativity bias" was observed. Classification performance tended to degrade for videos that induced low valence (unpleasantness) combined with high arousal. Intense negative emotions appear to generate neural signals that overwhelm the specific patterns associated with preference, making them harder for the model to distinguish.
Conclusion
By modeling the spatio-temporal connectivity of the brain using CNNs, this research demonstrates a significant leap in the accuracy of EEG-based preference recognition. Future work will focus on improving model robustness across individual differences and investigating the specific neural interference caused by high-arousal negative stimuli.
Summary updated