A brain network construction method for the assessment of functional quality of experience

Yifan Niu, Wei Tao, Yuan Zhang, Guangtao Zhai, Xia WuJournal of Image and Graphics 2024 · DOI

Abstract

Emerging video services introduce functional parameters that directly influence user cognitive states and experience quality, termed Functional Quality of Experience (fQoE). Electroencephalogram (EEG) signals provide rich information about brain activity, revealing patterns in brain networks that can serve as a reliable basis for fQoE assessment. This paper proposes a novel EEG-based brain network construction method to evaluate fQoE and investigate its underlying neural mechanisms. By manipulating functional parameters to induce varying fQoE levels, we collected EEG data and extracted multi-electrode features fused into graph structures. A self-attention graph pooling model was then used to identify fQoE-sensitive brain networks. Using bullet comment (danmaku) coverage as a case study, our method achieved classification accuracies between 80% and 86% across various video types. Results indicate that higher fQoE is associated with reduced brain connectivity in specific regions, providing a quantitative tool for optimizing video service parameters.

Introduction

Traditional Quality of Experience (QoE) research has primarily focused on technical transmission factors such as network latency, jitter, and video bitrate, often referred to as technical QoE (tQoE). However, modern video platforms offer a variety of functional features—such as interactive "bullet comments" (danmaku), virtual avatars, and personalized interfaces—that significantly impact user satisfaction. This dimension of user experience is defined as Functional Quality of Experience (fQoE).

While fQoE is critical for service optimization, it is traditionally measured through subjective surveys, which are prone to bias and lack physiological depth. This research addresses the need for an objective, neurophysiologically grounded assessment method by leveraging EEG signals to model the brain's response to functional video parameters.

Methodology

The proposed framework transforms raw neural activity into an interpretable assessment of user experience through a multi-stage pipeline.

EEG Data Preprocessing

To ensure high-quality signal analysis, raw EEG data undergoes rigorous preprocessing. This includes removing unusable electrodes, filtering power line interference, and applying Independent Component Analysis (ICA) to eliminate artifacts such as eye blinks. The continuous signal is then segmented using a sliding window approach to capture temporal dynamics.

The EEG data preprocessing pipeline, including artifact removal and data slicing.

Brain Network Construction

The core of the method lies in modeling the brain as a graph $G = (V, E)$.

  • Nodes ($V$): Represent individual EEG electrodes. Node features are derived from the Power Spectral Density (PSD) across five frequency bands ($\delta, \theta, \alpha, \beta, \gamma$).
  • Edges ($E$): Represent functional connectivity between brain regions, calculated using Pearson correlation coefficients.

A Graph Convolutional Network (GCN) is employed to capture the spatial topology of these networks. A key innovation is the Self-Attention Graph Pooling (SAGPool) mechanism, which automatically identifies the most relevant brain regions for fQoE classification, enhancing both performance and interpretability.

Experimental Design and Dataset

The study utilized "Bullet Chatting Videos" (BCV) from Bilibili to induce different fQoE levels. Fifty healthy participants watched videos across five categories (Auto-tune, Technology, Dance, Film, and Music) while their EEG signals were recorded using a 128-lead system.

The primary functional parameter manipulated was the danmaku coverage rate, set at five levels: 0%, 25%, 50%, 75%, and 100%. This setup allowed researchers to observe how varying levels of visual clutter and information density directly affect neural patterns.

The experimental paradigm for EEG dataset construction, showing the stimulus materials and signal acquisition setup.

Results and Discussion

The proposed model demonstrated superior performance in classifying fQoE levels compared to standard benchmarks like EEGNet.

Classification Performance

The model achieved high accuracy across all video types:

  • Auto-tune (Guichu): 86%
  • Music: 84%
  • Film: 82%
  • Technology: 81%
  • Dance: 80%

Neurophysiological Insights

The analysis revealed that higher fQoE (better experience) is correlated with a decrease in functional connectivity in the frontal pole, middle frontal gyrus, parietal, and temporal lobes. This suggests that excessive functional elements, like dense bullet comments, increase cognitive load, forcing the brain to increase its connectivity to process competing visual and linguistic information. Conversely, a "cleaner" and more satisfying experience allows for more efficient neural processing with fewer active connections.

Furthermore, features from the $\beta$ and $ \gamma$ bands were found to be the most discriminative, confirming that fQoE involves high-level cognitive processes such as active concentration and emotional integration.

Conclusion

This research provides the first EEG-based brain network construction method specifically designed for fQoE assessment. By identifying the neural markers of functional satisfaction, the method offers a quantitative, objective tool for video service providers to optimize their functional parameters. Future work will focus on real-time assessment and improving model robustness across diverse user populations.

Summary updated

Abstract

目的新兴视频服务的功能参数设置将会直接影响到用户的认知状态,进一步影响用户体验质量,称为功能性用户体验质量(functional quality of experience, fQoE)。脑电信号蕴含丰富的大脑活动信息,能够揭示复杂脑活动中的脑网络模式,为fQoE提供可靠的评估依据。为此,本文首次提出了一个基于脑电的脑网络构建方法以评估fQoE,并研究fQoE背后的神经机制。方法首先,通过改变功能参数诱发不同水平的fQoE,并同步收集脑电数据;然后,从脑电数据中提取单电极和多电极特征并以图结构进行融合,用以全面表征用户使用视频服务时的大脑状态;最后,使用基于自注意力图池化的脑网络构建模型来识别对fQoE敏感的脑网络,为fQoE提供可解释性,并进行分类以完成fQoE评估。结果本文以弹幕视频服务的弹幕覆盖率这一功能参数为例验证了方法的科学性和可行性。实验表明,提出的评估方法在多种视频类型的fQoE评估中均达到了满意的效果,最佳识别准确率分别为86%(鬼畜类)、81%(科技类)、80%(舞蹈类)、82%(影视类)和84%(音乐类)。结论来自fQoE相关的脑网络分析结果表明,额极、额中回、顶叶和颞叶的脑连接数量减少预示着观看弹幕视频的fQoE更高,即观看体验更好,同时也证明了功能参数通过影响人的脑状态进一步导致了fQoE的改变。本文的评估方法为fQoE的精确评估和视频服务功能参数的优化提供了来自神经生理学的定量工具和理论依据。
BibTeX
@article{niu2024brain,
  author = {Yifan Niu and Wei Tao and Yuan Zhang and Guangtao Zhai and Xia Wu},
  title = {{A brain network construction method for the assessment of functional quality of experience}},
  journal = {Journal of Image and Graphics},
  year = {2024},
  doi = {10.11834/jig.230500},
}
Figure