TL;DR
This paper introduces a modular NLP pipeline for automatically transferring linguistic annotations (like speaker attribution and quote detection) from English literary texts to German. By leveraging machine translation and sophisticated alignment techniques, they demonstrate a method to create valuable annotated literary corpora for low-resource languages, significantly reducing manual annotation effort.
What Problem Does This Paper Address?
Analyzing literary texts computationally, especially for tasks like character network analysis, often requires detailed annotations of elements like character mentions and quoted speech. Manually creating these annotations is extremely time-consuming and expensive. While automated methods exist for English, similar resources are scarce for many other languages, hindering cross-lingual literary analysis. This paper tackles the challenge of bridging this gap by proposing a system to automatically project existing English annotations onto German literary texts, using parallel corpora derived from translations.
Methods
The core of the paper is a modular NLP pipeline designed for annotation transfer. The pipeline consists of several key stages:
- Sentence Segmentation: Utilizes Stanford CoreNLP for robust sentence splitting, handling complexities of literary prose.
- Sentence Alignment: Employs a two-pass approach based on machine translation (MT) and BLEU scores. First, it finds 1:1 alignments maximizing BLEU similarity between MT-generated source translations and human translations. Second, it uses length-based heuristics and BLEU to align remaining sentences.
- Machine Translation: Leverages Nematus, a neural encoder-decoder model with attention, to translate English source texts into German.
- Word Alignment: Uses
fast_align, an unsupervised log-linear reparameterization of IBM Model 2, to align words between the source and target sentences. - Annotation Transfer: Directly transfers speaker and quote annotations based on the word alignments. The hypothesis is that quotation marks and proper names/pronouns will have high alignment precision.
A significant aspect of their methodology is the comparison between a fully automatic setting and a semi-automatic setting, where genre-dependent pre- and post-processing steps are applied to improve quality.
Datasets
- QuoteLi3 Corpus: Used as the source of English annotations. This corpus contains manual annotations of speakers and quotes in three novels: Emma, Pride and Prejudice (by Jane Austen), and The Steppe (by Anton Chekhov).
- Parallel Literary Corpora: The authors created parallel English-German corpora by using existing translations of Emma and Pride and Prejudice. They specifically used two different translations for Pride and Prejudice (PP_KS and PP_HS) to investigate the impact of translation choice.
Key Insights and Ideas
- Translation Choice Matters: The quality of the target language annotations is highly dependent on the chosen translation. Translations that are closer to the original text (as measured by BLEU similarity to an automatic translation) lead to significantly better sentence and word alignment, and consequently, better annotation transfer.
- Semi-Automatic Processing Boosts Quality: While a fully automatic pipeline is useful, applying genre-specific pre- and post-processing steps (like harmonizing punctuation or merging incorrectly split sentences) dramatically improves the quality of sentence alignment and subsequent annotation transfer.
- Direct Transfer is Viable for Quotes and Mentions: For specific annotation types like quotes (marked by quotation marks) and speaker mentions (often proper names), direct transfer based on word alignments can achieve high precision, especially when combined with semi-automatic processing.
- Translation Divergences are a Major Hurdle: Differences in sentence structure and phrasing between source and translated texts are a primary source of errors, particularly in sentence alignment, which can propagate to word alignment and annotation transfer.
Key Results
- Sentence Alignment Recall: Preprocessing the sentence-segmented XML input can increase recall from below 50% to over 90% for the PP_KS translation, and to 96%-100% for other translations.
- Word Alignment Precision: In the semi-automatic setting, precision for transferring speaker mentions and quotes reaches the high nineties. For quotes, precision can exceed 98% for the better translation (PP_HS).
- Impact of Translation: The PP_HS translation of Pride and Prejudice consistently yielded better results across sentence alignment recall and word alignment precision for speaker mentions and quotes compared to the PP_KS translation.
- BLEU as a Proxy: BLEU similarity between a human translation and an automatically generated translation serves as a good indicator for determining the most suitable translation for annotation projection.
Limitations and Future Work
- Translation Divergences: The paper acknowledges that translation divergences are a significant source of errors, particularly in sentence alignment, leading to unaligned or incorrectly aligned sentences.
- Multi-Word Expressions: The direct transfer method struggles with multi-word expressions, and resolving these requires more sophisticated alignment techniques.
- Future Work: The authors suggest integrating modules for automatic resolution of multi-word alignments and null alignments. They also propose exploring more sophisticated annotation projection methods beyond direct transfer, such as graph optimization or machine learning, to improve accuracy. They aim to further explore the use of machine learning for more complex projection tasks.
Summary updated