Modeling User Context for Valence Prediction from Narratives

Aniruddha Tammewar, Alessandra Cervone, Eva-Maria Messner, Giuseppe RiccardiInterspeech 2019

TL;DR

This paper explores how to predict a user's emotional state (valence) by analyzing their personal narratives. They found that incorporating context from previous narratives significantly improves prediction accuracy compared to analyzing single narratives in isolation, suggesting that understanding a user's emotional history is key to understanding their current state.

What Problem Does This Paper Address?

Predicting a person's emotional state, specifically their valence (how positive or negative they feel), from their personal narratives holds significant promise for mental healthcare applications, such as early diagnosis and monitoring of mental health conditions. Previous work, like the Interspeech 2018 ComParE challenge, focused on predicting valence from short, 8-second audio fragments of a single narrative. This approach, however, missed the crucial contextual information embedded within an individual's longer, personal storytelling. This paper aims to bridge this gap by investigating how to leverage the context of multiple narratives from the same individual to achieve more accurate valence prediction, focusing solely on textual data for broader applicability.

Methods

The researchers employed a combination of classical machine learning (ML) and deep learning (DNN) approaches.

  • Feature Engineering: They utilized sentiment polarity scores (derived from textblob-de), GloVe word embeddings, and TF-IDF features. Crucially, they also incorporated "previous valence class" as a "gold feature" (i.e., actual self-reported valence from a prior narrative) to represent historical emotional context.
  • Classical ML (Linear SVM): They experimented with Linear SVM, concatenating document vectors (derived from word embeddings or TF-IDF) with other narrative-level features. To incorporate context, they concatenated vectors from multiple narratives.
  • DNN Architectures:
    • Sequence Tagging: This approach treated the prediction of valence for a sequence of narratives as a sequence tagging problem. A bidirectional RNN (GRU) with an attention mechanism encoded each narrative, and a unidirectional RNN then predicted valence for each narrative in sequence, allowing previous narratives to influence current predictions.
  • Context Pair: This method focused on using only the current and immediately preceding narrative to predict the current valence.

A key aspect of their approach was the decision to use only textual data, making their method applicable even when audio is unavailable.

Datasets

The primary dataset used is the Ulm State of Mind (USOMS) corpus. A subset of this corpus, specifically the "Self-Assessed Affect Sub-Challenge" data from Interspeech 2018, was utilized. This subset consists of personal narratives from 100 speakers (85 female, 15 male, aged 18-36). Each individual provided four narratives (two negative, two positive), each approximately 5 minutes long, with self-reported valence scores collected before and after each narrative. The authors preprocessed the data by removing punctuation, converting to lowercase, and filtering out users with incomplete narrative sets, ultimately working with data from 72 users (288 narratives).

Key Insights and Ideas

  • Context is King for Emotion: The most significant insight is that understanding a user's emotional history through their previous narratives dramatically improves the accuracy of predicting their current emotional state. This moves beyond analyzing isolated moments to understanding emotional trajectories.
  • Beyond Sentiment Words: The attention mechanisms in their DNN models revealed that not only direct sentiment-carrying words but also "emotion-triggering concepts" (like events, people, and places) and even disfluencies (like "uhm") in narratives play a crucial role in predicting valence. This suggests models can learn nuanced emotional associations.
  • Text-Only Approach is Viable: The success of their text-based models demonstrates that valuable emotional context can be extracted from written narratives alone, broadening the applicability of such systems beyond audio-dependent scenarios.
  • Temporal Order Matters: The sequence tagging approach, which explicitly models the temporal order of narratives, showed promising results, indicating that the progression of emotions and topics over time is informative.

Key Results

  • Linear SVM: Incorporating the "previous valence class" feature improved accuracy by nearly 9% over a baseline that used only the current narrative. Using TF-IDF features for context (current and previous narrative) also yielded a significant improvement.
  • DNN Architectures:
    • The baseline BiRNN with attention (narrative in isolation) achieved 58.2% accuracy.
  • The "sequence tagging" architecture improved accuracy to 61.8%.
  • The "context pair" method, using current and previous narratives, achieved the highest accuracy of 62.4%.
  • Qualitative Analysis: Visualizations of attention weights highlighted that concepts like "abi" (high school graduation exam) received attention across multiple narratives, demonstrating the model's ability to track recurring themes and their emotional impact. SVM feature importance also confirmed the role of events and disfluencies.

Limitations and Future Work

  • Limited Sample Size: The authors acknowledge that the results should be interpreted with caution due to the limited sample size (72 users) and the lack of longitudinal data.
  • Diversity: The dataset's demographic diversity (e.g., age, gender) could be expanded.
  • Future Directions: Replication studies with larger and more diverse sample sizes are recommended. Further research could explore more sophisticated context modeling techniques, investigate the impact of different types of narratives, and potentially integrate multimodal data (audio, text) if available. The authors also suggest that their approach holds promise for real-world applications in mental health care.

Summary updated

BibTeX
@inproceedings{tammewar2019modeling,
  author = {Aniruddha Tammewar and Alessandra Cervone and Eva-Maria Messner and Giuseppe Riccardi},
  title = {{Modeling User Context for Valence Prediction from Narratives}},
  booktitle = {Interspeech},
  year = {2019},
}
Figure