Affect detection and metaphor in e-drama

Li Zhang, John A. Barnden, Robert Hendley, Mark Lee, Alan M. Wallington, Zhigang WenInternational Journal of Continuing Engineering Education and Life-Long Learning 2008 · DOI

TL;DR

This paper introduces an affect-detection module for an e-drama system, aiming to automate directorial functions and enable more dynamic virtual characters. By analyzing textual "speeches" for emotional cues, including metaphorical expressions, the system can better stimulate improvisations and create more engaging interactive narratives.

What Problem Does This Paper Address?

Improvised drama, or e-drama, is a valuable tool for education, counseling, and conflict resolution, allowing participants to engage in role-playing within virtual environments. However, managing these improvisations, especially with multiple actors and virtual characters, places a significant burden on human directors. Directors must monitor interactions, ensure characters stay in character, and intervene when emotions or plotlines deviate. This paper addresses the challenge of automating some of these directorial functions to make e-drama systems more usable and to create more intelligent, responsive virtual characters. Specifically, it focuses on detecting affect (emotions, moods, rudeness, value judgments) in character dialogue to inform automated responses and directorial suggestions. A key challenge is the unique, often ungrammatical and abbreviated language used in e-drama sessions, which requires specialized processing.

Methods

The core of this work is an affect-detection module designed to analyze textual "speeches" from characters in the e-drama system. The system employs a blackboard architecture where various knowledge sources perform analyses and contribute to a central data structure. Key processing steps include:

  • Pre-processing: This is crucial due to the informal and often ungrammatical language used in e-drama. It involves handling misspellings, abbreviations (using a lookup table and part-of-speech tagging for disambiguation), slang, and textese.
  • Imperative Mood Detection: The system identifies imperative sentences, as these often convey strong emotions or rude attitudes. It uses the Rasp parser and semantic profiles to distinguish between imperative and declarative moods, especially in ambiguous cases.
  • Pattern Matching: A rule-based Java framework (Jess) implements pattern-matching rules to identify keywords, phrases, and partial sentence structures indicative of affect. This approach is designed to be robust to variations in language.
  • Semantic Analysis: Beyond simple pattern matching, the system incorporates semantic analysis, including the use of WordNet and semantic profiles, to understand the meaning and affective implications of utterances, particularly for first-person statements.
  • Metaphorical Expression of Affect: A distinctive feature is the focus on detecting affect conveyed through metaphor. This involves identifying stock phrases and exploring more creative metaphorical expressions of emotion.

The output of the affect detection is an affect interpretation, which is then used to:

  1. Control an automated actor (EMMA) to generate appropriate responses.
  2. Provide suggestions to the human director.
  3. Drive animations of avatars to reflect detected affect.

Datasets

The paper describes analysis performed on a corpus of e-drama sessions, specifically from improvisations based on a school bullying scenario and a Crohn's disease scenario. These sessions involved school children aged 8-12. The authors also mention using a dictionary of the 1,000 most frequently used English words (Heise, 1965) for semantic profiles and WordNet for lexical semantic extraction. No new datasets were explicitly introduced for benchmarking.

Key Insights and Ideas

  • Language in e-drama is a significant challenge: The informal, ungrammatical, and abbreviated nature of e-drama dialogue requires specialized pre-processing and analysis techniques beyond standard NLP tools.
  • Affect is paramount: In e-drama, the literal meaning of statements is often secondary to the affective state being expressed, making affect detection critical for understanding and responding.
  • Metaphor is a key vehicle for affect: The paper highlights that affect is frequently conveyed implicitly through metaphors, and developing methods to detect this is crucial for a comprehensive understanding.
  • Automating directorial functions is feasible: The work demonstrates that even limited affect detection can be sufficient to stimulate improvisation and create more engaging interactions, reducing the burden on human directors.
  • User perception is surprisingly robust: User testing revealed that participants often didn't notice when a bit-part character was played by an automated agent, suggesting that the automated character's contributions were perceived as natural enough.

Key Results

  • The developed affect-detection module can identify affective aspects like emotions, moods, rudeness, and value judgments in character speeches.
  • The system's automated actor (EMMA) can generate responses that are "reasonably appropriate to the discourse," stimulating improvisation.
  • User testing indicated that the presence of an automated bit-part character (Dave) did not significantly impact user engagement or enjoyment compared to a human-played character.
  • While inter-annotator agreement among human coders was moderate, the system showed some promise in its ability to detect affect, with potential for improvement through further refinement.

Limitations and Future Work

  • Limited scope of affect detection: The authors acknowledge that they do not purport to detect all types of affect or to do so with high reliability. The focus is on practical processing techniques and theoretical investigation.
  • Ambiguity in language: The highly idiosyncratic and ungrammatical nature of e-drama language presents ongoing challenges for accurate analysis.
  • Metaphorical complexity: While the paper emphasizes the importance of metaphorical affect, the implementation of metaphor processing is still in its early stages.
  • Future Work: The authors plan to further develop their theoretical ideas about metaphor and implement more sophisticated metaphor processing to extract additional affective hints. They also aim to make the parameter 'N' (controlling response frequency) dynamically adjustable and explore more advanced methods for affect detection and response generation.

Summary updated

Abstract

We report work on adding affect-detection to an existing e-drama programme, a text-based software system for (human) dramatic improvisation in simple virtual scenarios, for use primarily in learning contexts. The system allows a human director to monitor improvisations and make interventions, for instance in reaction to excessive, insufficient or inappropriate emotions in the characters' speeches. Within an endeavour to partially automate directors' functions, and to allow for automated affective bit part characters, we have developed an affect-detection module. It is aimed at detecting affective aspect (concerning emotions, moods, rudeness, value judgments, etc.) of human-controlled characters' textual 'speeches'. The work also accompanies basic research into how affect is conveyed linguistically. A distinctive feature of the project is a focus on the metaphorical ways in which affect is conveyed. The project addresses the special issue themes such as making interactive narrative learning environments more usable, building them, and supporting reflection on narrative construction.
BibTeX
@article{zhang2008affect,
  author = {Li Zhang and John A. Barnden and Robert Hendley and Mark Lee and Alan M. Wallington and Zhigang Wen},
  title = {{Affect detection and metaphor in e-drama}},
  journal = {International Journal of Continuing Engineering Education and Life-Long Learning},
  year = {2008},
  doi = {10.1504/ijceell.2008.017378},
}
Figure