Multi-level Contrastive Learning for Script-based Character Understanding

Dawei Li, Hengyuan Zhang, Yanran Li, Shiping Yang

Understanding character identities and personalities within long-form scripts remains a significant challenge for modern language models. Unlike standard narratives, scripts are primarily composed of multi-party conversations where character traits are revealed subtly over thousands of pages. This paper introduces a multi-level contrastive learning framework designed to align character representations across dialogue, scene summaries, and different script segments, significantly improving performance on tasks like character guessing and coreference resolution.

The Challenge of Script-based Understanding

Extracting a consistent "personality" from fragmented dialogue is inherently harder than reading descriptive prose. Scripts present two primary hurdles:

  1. Text Type Complexity: Scripts involve complex, multi-party interactions where traits are often implied rather than stated.
  2. Global Context Requirements: Scripts can span millions of words. Most Pre-trained Language Models (PLMs) have limited context windows, causing them to fail at connecting a character's behavior across distant scenes (e.g., Scene A and Scene Z).

Methodology

The proposed framework treats character understanding as a representation alignment problem, utilizing three distinct loss functions during training to build robust character embeddings.

The multi-level contrastive learning architecture, illustrating the alignment of summary and conversation encodings through cross-sample and summary-conversation losses.

Contrastive Loss Functions

  • Summary-Conversation Contrastive Loss ($L_{Sum}$): By treating scene summaries and actual dialogue as two different "views" of the same character, the model learns to associate high-level character descriptions with specific conversational styles.
  • Cross-Sample Contrastive Loss ($L_{Cross}$): To overcome context window limitations, the model aligns representations of the same character across different samples in a training batch. This incentivizes the creation of consistent embeddings regardless of the specific scene context.
  • Two-Stage Training: The model is first trained with both contrastive and supervised losses to build a character-aware foundation, followed by fine-tuning on the specific target task.

Key Insights and Results

The framework was evaluated on benchmarks including TVSHOWGUESS (character identification in shows like Friends) and Character Identification Datasets (coreference resolution).

The "Global" Advantage

A major finding is that character identity is a global property. By using cross-sample contrastive learning, the model effectively "remembers" a character across context boundaries. This is visually evident in the improved clustering of character embeddings.

Visualization of character embeddings for Longformer-P-Large. (a) shows the baseline, while (b) demonstrates the tighter, more distinct character clusters achieved with the proposed method.

Performance Highlights

  • Character Guessing: On the Longformer-P-large backbone, the method improved Macro-F1 from 75.92 to 76.52.
  • Character Linking: Reached a Macro-F1 of 83.7 on SpanBERT-large, a notable jump over the 82.8 vanilla baseline.
  • LLM Comparison: Surprisingly, ChatGPT-3.5 performed the worst among all baselines. Despite its scale, it struggled with the global nature of script understanding and the nuances of character-guessing, likely due to limited context for few-shot demonstrations.

Limitations and Future Work

While effective, the $L_{Sum}$ loss currently depends on high-quality summaries. Although ChatGPT-generated summaries can serve as a substitute, the method relies on auxiliary descriptive text. Future research may explore integrating this framework into larger open-source models (7B+ parameters) and applying these alignment strategies to other long-form domains like legal document analysis or deep fiction studies.

Summary updated

Abstract

In this work, we tackle the scenario of understanding characters in scripts, which aims to learn the characters' personalities and identities from their utterances. We begin by analyzing several challenges in this scenario, and then propose a multi-level contrastive learning framework to capture characters' global information in a fine-grained manner. To validate the proposed framework, we conduct extensive experiments on three character understanding sub-tasks by comparing with strong pre-trained language models, including SpanBERT, Longformer, BigBird and ChatGPT-3.5. Experimental results demonstrate that our method improves the performances by a considerable margin. Through further in-depth analysis, we show the effectiveness of our method in addressing the challenges and provide more hints on the scenario of character understanding. We will open-source our work on github at https://github.com/David-Li0406/Script-based-Character-Understanding.
BibTeX
@inproceedings{li2023multi,
  author = {Dawei Li and Hengyuan Zhang and Yanran Li and Shiping Yang},
  title = {{Multi-level Contrastive Learning for Script-based Character Understanding}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2023},
  url = {https://openreview.net/forum?id=uACH81RKph9},
}
Figure