EE-Motion: An EEG-based Web Architecture for Live Affective Music Recommendation

Sabino Roccotelli, Potito Aghilar, Vito Walter Anelli, Alejandro Bellogín, Tommaso Colafiglio, Michelantonio Trizio, Tommaso Di NoiaarXiv 2026 · arXiv

TL;DR

EE-Motion is a novel web architecture that uses real-time EEG signals from consumer headbands to steer music recommendations based on a listener's shifting emotional state. By introducing a model-agnostic "affective adapter," the system can serve anonymous users and swap between different recommendation backbones—like graph neural networks or transformers—without the need for expensive model retraining.

What Problem Does This Paper Address?

Music is deeply emotional, and our moods aren't static; they drift throughout a listening session. However, most modern recommender systems (RecSys) are "emotionally blind." They typically rely on discrete logs (clicks, plays) and static user profiles built from historical data.

This creates three major bottlenecks:

  1. Infrastructure: Standard RecSys aren't built to handle high-frequency, noisy physiological telemetry (like EEG) directly in the browser.
  2. The "Cold Start" for Affect: Most affect-aware models require offline training on specific users. If an anonymous newcomer shows up, the system can't easily incorporate their live emotional state without retraining.
  3. BCI Isolation: Brain-Computer Interfaces (BCIs) have traditionally required specialized third-party drivers, making them difficult to integrate into the standard web ecosystem.

Methods

The authors propose a decoupled microservice architecture that bridges the gap between raw brainwaves and recommendation logic.

Figure 1: Overview of the Web-based Live Affective Architecture. The browser acquires raw EEG via Web Bluetooth and streams it to the bci-processor for V/A mapping; the affective adapter then scores candidates across swappable WarpRec backbones.

The Pipeline:

  • Acquisition: Using a consumer Muse 2 EEG headband and the Web Bluetooth API, the system streams raw 4-channel EEG data directly to the browser.
  • BCI-Processor: This service filters the noisy signal and extracts two key features: Frontal Alpha Asymmetry (FAA) (a proxy for valence/positivity) and the Arousal Index (AI). These are mapped onto Russell’s circumplex plane—a 2D space representing emotional state.
  • The Affective Adapter: This is the core innovation. It acts as a bridge, allowing the system to project an anonymous user into a latent space by averaging the embeddings of a few "favorite tracks" they select at the start. It then uses the live Valence/Arousal (V/A) coordinates to re-rank music candidates in real-time.
  • WarpRec Integration: The system uses the WarpRec framework to host swappable models, specifically testing ACE (a graph-based model) and SASRec (a self-attentive sequential model).

Datasets

The system was evaluated using the Lastfm-VADS catalog. This dataset is particularly useful because it provides inferred item-level V/A profiles, allowing the recommender to match the user's live EEG state to the emotional "fingerprint" of specific songs.

Key Insights and Ideas

  • Input Swappability: Because the architecture decouples signal processing from ranking, the "affective signal" is treated as a generic coordinate. This means you can swap the EEG headband for a manual 2D cursor (for users without hardware) without changing a single line of the recommendation code.
  • Bypassing User Tables: By parameterizing users through a small set of favorite tracks rather than persistent IDs, the system solves the "anonymous newcomer" problem. It treats the user's state as a dynamic combination of their initial preferences and their current physiological drift.
  • Real-time Steering, Not Recognition: The authors wisely frame the EEG data as a "control signal" to steer recommendations rather than a clinical tool for "emotion recognition." This lowers the bar for accuracy while still providing a responsive, interactive experience.

Key Results

The researchers performed quantitative profiling to ensure the system could keep up with the 8 Hz EEG stream (a 125ms window).

  • Latency: As shown in Figure 2, the end-to-end cycle time for both ACE (~90ms) and SASRec (~80ms) stays well below the 125ms deadline, preventing any queuing delays or "laggy" recommendations.
  • Recommendation Quality: Using a Wilcoxon signed-rank test, they found no statistically significant difference in recommendation quality (nDCG@10) between the live affective version and the offline counterparts. This proves that the "affective adapter" successfully infuses live signals without degrading the underlying model's performance.
Figure 2: End-to-end latency breakdown and execution trace of a live recommendation cycle for ACE and SASRec. The recommender microservice dominates overall latency, while total cycle time remains strictly below the 125 ms deadline of the 8 Hz EEG stream, preventing queuing delays.

Limitations and Future Work

  • Proxy vs. Reality: The authors acknowledge that FAA and AI are "physiological proxy indices" rather than validated emotional states. Future work could involve more sophisticated deep-learning models for EEG feature extraction.
  • Cold Start Depth: While the "favorite tracks" method works for anonymous users, the system's understanding of a user is still relatively shallow compared to long-term historical profiles.
  • Privacy: While the current system processes data ephemerally in memory, scaling such a system would require rigorous ethical frameworks regarding the storage and use of neural data.

For those interested in seeing the system in action, the authors provided a walkthrough of the UI, showing the transition from device pairing to live V/A tracking on the circumplex plane.

Figure 4: Recommendation page: top-10 V/A-conditioned tracks with audio playback and ratings. The plane displays live V/A proxy coordinates, with on-demand re-ranking.

Summary updated

Abstract

Music listening is intrinsically tied to emotion, which drifts continuously within a single session. Although modern browsers interface with wearable brain-computer interfaces, recommendation systems remain structured around discrete logs and lack the infrastructure to integrate continuous physiological signals. Furthermore, existing affect-aware models rely on static user profiles and offline training, failing to serve anonymous newcomers without model retraining. We present a web platform that resolves both challenges. Pairing with a consumer electroencephalogram (EEG) headband over Web Bluetooth, the system captures live EEG, mapping frontal alpha asymmetry and arousal index to normalized valence-arousal proxy coordinates on Russell's circumplex plane. Driven by these coordinates, an affective adapter unifies structurally diverse recommendation backbones under a shared live scoring mechanism. By parameterizing anonymous visitors through a few favorite tracks, the module recommends tracks conditioned on live proxy coordinates. This demonstrates that real-time EEG-derived control signals can steer recommendations without persistent user tables or model retraining.
BibTeX
@misc{roccotelli2026ee,
  author = {Sabino Roccotelli and Potito Aghilar and Vito Walter Anelli and Alejandro Bellogín and Tommaso Colafiglio and Michelantonio Trizio and Tommaso Di Noia},
  title = {{EE-Motion: An EEG-based Web Architecture for Live Affective Music Recommendation}},
  year = {2026},
  eprint = {2611.21032},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.21032},
}
Figure