Introduction
In computational literary analysis, distinguishing between a narrator's voice and character dialogue is a fundamental pre-processing step. This task, known as Automatic Annotation of Direct Speech (AADS), enables downstream applications like character network mapping and persona modeling. For French literature, AADS is particularly challenging due to "incises"—short narrative interruptions like "dit-il" (said he) inserted within spoken sentences—and the often-poor formatting of digitized historical texts.
Researchers from Deezer and EPFL have developed a unified framework for AADS in French, consolidating a large-scale narrative dataset and benchmarking deep learning models against traditional rule-based systems.
Datasets
The study utilizes a consolidated dataset of 86 chapters (approximately 680,000 tokens) from French novels published between 1830 and 1937. The data is divided into a "Clean Set" of well-formatted texts and a "Noisy Set" (Test$_N$) derived from the Straight Talk! corpus, which mimics the formatting challenges found in real-world digitized archives.
Methodology
The authors compared three primary approaches:
- Regex (Rule-based): A baseline utilizing standard typographic cues such as guillemets (« ») and dialogue dashes (—).
- BiLSTM-CRF: A bidirectional sequence labeling architecture designed to capture contextual dependencies.
- Fine-tuned CamemBERT: A state-of-the-art transformer model pre-trained specifically on French text.
Performance was measured using Token-level F1-scores and Strict Sequence Matching (SSM) F1-scores. Additionally, the researchers employed ZoneMap Error (ZME) to categorize model failures into misses, false alarms, splits, and merges.
Key Results and Insights
The evaluation revealed a stark contrast between rule-based and deep learning approaches. While the Regex baseline performed well on clean text (90% Token F1), its performance plummeted to 47% on the noisy dataset. In contrast, CamemBERT demonstrated significant robustness, maintaining a 93% Token F1 even on noisy text.
The "incise" challenge remains the most difficult aspect of AADS. Deep learning models, particularly transformers, are significantly better at "jumping over" these interruptions to maintain speech span integrity. The authors also found that a simple "Clause-Consistency" post-processing step—forcing consistency within punctuation-defined clauses—could boost sequence-level scores by up to 8 points.
Conclusion and Future Work
Fine-tuned transformers like CamemBERT are the most effective tools for AADS in French, offering the robustness necessary to handle the linguistic and structural complexities of historical literature. Future research will focus on transitioning from simple speech detection to full speaker attribution and addressing the challenge of back-to-back dialogue turns without narrative markers.
Summary updated