Automatic Quote Attribution in Chinese Literary Works

Xingxing Yang, Yu WangACL 2024

TL;DR

This paper introduces a novel pipeline for automatically attributing quotes to speakers in Chinese literary works, tackling challenges like coreference and distinguishing between dialogue, crowds, and soliloquies. Their approach combines extractive QA, multi-choice QA, and co-reference resolution, achieving promising results on a dataset of modern Chinese novels.

What Problem Does This Paper Address?

Understanding who is speaking in a literary text is crucial for comprehending narratives. While manual attribution is feasible for small texts, it becomes a bottleneck for large-scale literary analysis. This paper addresses the challenge of automatically attributing spoken quotes to their correct speakers in Chinese literary works. This is particularly complex due to:

  • Coreference: Characters can be referred to by proper names, pronouns (he/she/I), or descriptive phrases (e.g., "the detective").
  • Ambiguous Speech: Quotes might not be from a single individual but from crowds, sound effects, or internal monologues (soliloquies).
  • Conversational Flow: Dialogues often follow patterns where speakers aren't explicitly named in every turn, making it hard to track who said what.
  • Language Specifics: Chinese literature presents unique challenges, including the use of different quotation marks and the prevalence of certain linguistic structures.

Methods

The authors propose a multi-stage pipeline for quote attribution:

  1. Quote Extraction: They start by using regular expressions to extract quoted text, noting that not all Chinese texts strictly adhere to standard quotation marks. They also developed a rule-based filter to identify and exclude sound effects (e.g., "Huhu" followed by "声").
  2. QA-based Quote Attribution: For most cases, they frame quote attribution as an extractive question-answering (QA) task. A BERT model is fine-tuned to identify and extract explicit speaker names from the text. They handle the pronoun "我" (I) as a special case, treating it as a character name due to its consistent reference in first-person narratives.
  3. MC-based Quote Attribution: To address performance degradation on longer conversational threads, they introduce a multi-choice QA approach. This model is trained to select the correct speaker from a predefined set of choices, improving accuracy in complex dialogue sequences.
  4. Co-reference Resolution: To resolve pronoun references (like "he" or "she"), they employ a co-reference resolution model. This is crucial for linking pronouns to the correct speaker, especially when the actual name is distant from the quote. They use a Chinese pre-trained Longformer model and Jieba for word segmentation.
  5. Classification of Crowds, Soliloquies, and Dialogues: A hybrid approach is used to distinguish between these categories. A rule-based method identifies keywords associated with crowds, while a BERT-based classifier handles the broader classification task.

Datasets

The primary dataset used is the CLD (Chinese Literary Dialogue) dataset, comprising 1,991 modern Chinese novels. This dataset was annotated by literature practitioners and includes main character information, novel texts, and attributed quotes. The authors also mention constructing specific datasets for each task (QA, MC-QA, co-reference) based on this original corpus.

Key Insights and Ideas

  • Framing Quote Attribution as QA: Treating quote attribution as a question-answering problem, especially extractive QA, proves to be a robust and effective approach for identifying explicit speakers.
  • Multi-Choice QA for Conversational Threads: The performance drop on long dialogues highlights the limitations of standard models. The multi-choice QA approach offers a practical solution by constraining the answer space.
  • Hybrid Classification for Speech Types: Combining rule-based filtering with BERT classification for crowds, soliloquies, and dialogues provides a more nuanced way to handle different utterance types beyond simple speaker attribution.
  • Importance of Co-reference Resolution: Accurately resolving pronouns is vital, especially in literary texts where characters might be referred to indirectly over long passages.

Key Results

  • Extractive QA: Achieved an F-score of 95.1794% and EM of 93.1452% for single-paragraph contexts. For mixed single- and multi-paragraph contexts, the F-score was 88.2331%.
  • MC-based Quote Attribution: The UniMC model achieved an accuracy of 0.9259.
  • Co-reference Resolution: The Fast-coref model achieved an F-score of 95.4%.
  • Classification: BERT classification for crowds achieved an accuracy of 0.8458, outperforming the rule-based method (0.6308) for this specific task. The overall BERT classification for crowds, soliloquies, and dialogues achieved 0.72.

The authors note that individual modules perform well, indicating the potential of their pipeline.

Limitations and Future Work

Limitations:

  • Edge Cases: The study neglects irregular speech content without explicit quotes in Chinese novels. Audio effects are handled by simple rules, but these rules are not always applicable.
  • Language Specificity: The current study is limited to modern Chinese literature. While the methodology might be applicable to other languages, language differences could pose challenges.
  • Longer Contexts: Performance drops when dealing with very long contexts, and resolving co-reference for names spanning multiple chapters remains an unexplored challenge.
  • Model Chaining: The pipeline's reliance on a chain of models means that errors in earlier stages can propagate and affect subsequent predictions.

Future Work:

  • Robust Annotated Dataset: Developing a comprehensive, well-annotated dataset for quote attribution in fiction, covering diverse writing styles and genres.
  • Leveraging LLMs: Exploring the integration of Large Language Models (LLMs) for more direct and accurate speaker identification, potentially reducing the need for multiple cascaded models. LLMs could also offer more flexibility in output formatting.

Summary updated

BibTeX
@inproceedings{yang2024automatic,
  author = {Xingxing Yang and Yu Wang},
  title = {{Automatic Quote Attribution in Chinese Literary Works}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
}
Figure