Sentence-Alignment in Semi-parallel Datasets

Steffen Frenzel, Manfred StedeNAACL 2025

TL;DR

This paper tackles the challenging task of aligning sentences in "semi-parallel" texts – versions of the same document that have been altered, not just translated. They propose and evaluate two hypotheses: that aligning within a "context window" improves efficiency and quality, and that segmenting texts into Elementary Discourse Units (EDUs) can enhance alignment for complex texts.

What Problem Does This Paper Address?

Sentence alignment is a crucial preprocessing step for many NLP tasks, traditionally applied to parallel texts (direct translations). However, real-world scenarios often involve "semi-parallel" texts, where one version of a document has been modified. Examples include simplified language versions for accessibility, or different editions of academic works. Aligning these texts is difficult because the sentence structures and wording can vary significantly, making direct word-for-word or sentence-for-sentence matching unreliable. This paper aims to develop and evaluate methods for robust sentence alignment in these complex, monolingual semi-parallel datasets.

Methods

The authors explore two main hypotheses through their experimental setup:

  1. Context Window Alignment: Instead of comparing every sentence pair across two documents, they hypothesize that matching sentences are likely to be found in close proximity. They implement a "context window" approach where the search for a match for a sentence in one text is limited to a defined window of sentences in the other text. This is expected to improve efficiency and potentially accuracy by focusing the search.
  2. Elementary Discourse Unit (EDU) Segmentation: Recognizing that complex sentences can be hard to align, they propose segmenting texts into EDUs. EDUs are defined as the smallest units that convey a complete proposition. By aligning at this finer granularity, they expect to achieve more precise alignments, especially for heavily altered texts. They use a modified DPLP parser, trained on a German corpus, to perform this segmentation.

For sentence embeddings, they experimented with two models: T-Systems-onsite/cross-en-de-roberta-sentence-transformer and paraphrase-multilingual-mpnet-base-v2. The RoBERTa-based model consistently yielded better results and was chosen for the final experiments. They compare two alignment strategies: a "paraphrase function" that considers all possible unit pairs and a "context window" function that limits the search.

Datasets

The paper utilizes three distinct German semi-parallel datasets:

  • Hannah Arendt essays: Two versions of an essay on Franz Kafka, one original and one a radio broadcast. This dataset is characterized by long, complex sentences and significant alterations, making it the most challenging.
  • GDR literature encyclopedias: Pairs of encyclopedia entries about authors from the former German Democratic Republic, sourced from different encyclopedias. This dataset is notable for its terse style and sometimes abbreviated sentences, making it suitable for testing sub-sentence level alignment.
  • Plain language dataset: Original and simplified versions of news reports. This dataset is more parallel and contains shorter, less complex sentences compared to the Arendt essays.

Key Insights and Ideas

  • Context is King (for Efficiency): The "context window" approach significantly speeds up alignment compared to comparing all possible sentence pairs, especially for the encyclopedia data. This highlights the practical advantage of leveraging positional information.
  • EDU Segmentation's Mixed Bag: While EDUs offer a finer granularity, aligning at the EDU level didn't consistently outperform sentence-level alignment across all datasets. The benefit seems to depend on sentence complexity and the degree of parallelism. For very complex and less parallel texts like the Arendt essays, the advantage of EDUs is less pronounced, possibly due to the sheer difficulty of the alignment task itself.
  • "Meaning-Heavy" Terms are Tricky: For highly altered texts like the Arendt essays, common "meaning-heavy" terms (names, nouns) can lead to similar sentence embeddings, making it difficult for models to distinguish between them and leading to incorrect alignments. This points to a limitation in the current embedding models' ability to capture subtle semantic differences.
  • Multi-alignment and Thresholds Matter: Allowing multiple alignments for a single element and using a cosine similarity threshold generally improved alignment quality, suggesting a more flexible approach is beneficial.

Key Results

  • Context Window Dominance: The "context window" alignment algorithm generally outperformed the "paraphrase function" in terms of both alignment quality (measured by Cohen's Kappa) and computing time. It was faster by an average of 14%, with significant speedups for the encyclopedia data.
  • Sentence-Level Advantage: For the Arendt essays and plain language data, sentence-level alignment achieved slightly better results (higher Cohen's Kappa) than EDU-level alignment. For example, on the Arendt essays, sentence-level alignment achieved a Cohen's Kappa of 0.65, while EDU-level alignment was 0.49.
  • GDR Encyclopedias Shine: This dataset yielded good results, with Kappa scores ranging from 0.65 to 0.75, indicating that the alignment methods are effective even for texts with abbreviated sentences.
  • Arendt Essays Remain Challenging: Despite improvements, the Arendt essays dataset presented the most difficulties, with a higher proportion of unaligned segments and lower Kappa scores, underscoring the impact of significant text alterations.

Limitations and Future Work

  • Handling Heavily Altered Texts: The paper acknowledges that for heavily altered texts, the context window size needs to be significantly increased to find correct matches, diminishing the efficiency advantage. They suggest implementing a "previous paragraph alignment" as a potential solution before sentence alignment.
  • EDU Segmentation Nuances: The effectiveness of EDU segmentation is still being explored. Future work should include more data that offers a compromise between the simplicity of plain language texts and the complexity of the Arendt essays. Additionally, exploring and fine-tuning more EDU segmentation models is recommended.
  • Embedding Quality: A key limitation identified is the inability of current sentence embeddings to clearly distinguish similar but non-identical phrases, especially in "meaning-heavy" texts. Future research should focus on developing more precise embeddings that can better capture these subtle semantic differences.
  • Further Model Exploration: The authors plan to investigate additional models for EDU segmentation and further check the quality of sentence embeddings.

Summary updated

BibTeX
@inproceedings{frenzel2025sentence,
  author = {Steffen Frenzel and Manfred Stede},
  title = {{Sentence-Alignment in Semi-parallel Datasets}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure