Sentence Alignment for Monolingual Comparable Corpora

Regina Barzilay, Noemie ElhadadEMNLP 2003 · OpenReview

Abstract

We address the problem of sentence alignment for monolingual corpora, a phenomenon distinct from alignment in parallel corpora. Aligning large comparable corpora automatically would provide a valuable resource for learning of text-to-text rewriting rules. We incorporate context into the search for an optimal alignment in two complementary ways: learning rules for matching paragraphs using topic structure and further refining the matching through local alignment to find good sentence pairs. Evaluation shows that our alignment method outperforms state-of-the-art systems developed for the same task.

Introduction

In Natural Language Processing (NLP), parallel corpora—direct translations of the same text—have long been the gold standard for training translation models. However, "comparable corpora," which consist of texts in the same language describing the same events or concepts in different styles (e.g., expert vs. layperson), are far more abundant. These corpora are essential for tasks like text simplification and summarization.

The primary challenge lies in the "noisy" nature of comparable corpora. Unlike translations, the information order can vary significantly, and many sentences may lack a direct equivalent. Traditional alignment tools often fail because they assume linear progression or rely on simple lexical overlap, which cannot distinguish between true paraphrases and unrelated sentences sharing common words.

Proposed Methodology

The researchers propose a hierarchical approach that leverages the topical structure of documents to guide sentence alignment. This method moves beyond simple word-matching by considering the context in which a sentence appears.

1. Topical Structure Induction

The system first identifies recurring "topics" at the paragraph level across the entire collection. Using hierarchical clustering, it groups paragraphs that convey similar types of information, such as "history" or "geography" in city descriptions.

2. Macro Alignment

Using a small training set, the system learns mapping rules between these clusters. For instance, it might learn that a "symptoms" paragraph in a technical medical text typically corresponds to a "symptoms" paragraph in a simplified version.

Hierarchical alignment approach using topical structure and paragraph mapping.

3. Micro Alignment

By mapping paragraphs first, the search space for sentence alignment is drastically reduced. The system then employs a modified version of the Needleman-Wunsch algorithm to find the best sentence-level matches within those mapped paragraphs, allowing for many-to-many alignments.

Experimental Setup

The method was evaluated using a dataset compiled from Encyclopedia Britannica and Britannica Elementary.

  • Data: 103 pairs of city descriptions (e.g., "Lisbon," "Jakarta").
  • Characteristics: The elementary version is a simplified subset but often contains updated or distinct details.
  • Split: 92 pairs were used for training/clustering, and 11 pairs were reserved for a human-annotated test set.

Results and Discussion

The evaluation demonstrates that context is a more reliable indicator of a match than lexical similarity alone. A simple cosine similarity measure, when constrained by topical context, outperformed sophisticated systems like SimFinder.

Precision-Recall curves comparing the proposed method against SimFinder and cosine similarity baselines.

At a fixed recall level of 55.8%, the "Full Method" achieved a precision of 76.9%, significantly higher than SimFinder (24.0%) and the cosine baseline (57.9%). When provided with "ideal" manual clusters, precision reached 80.9%, indicating that while the unsupervised clustering is effective, there is still room for improvement.

Conclusion

This hierarchical alignment method exploits the inherent topical regularity of specific domains. While it excels in structured environments like encyclopedias, its performance in highly heterogeneous collections remains an area for future research. Ultimately, this approach paves the way for automatically inducing transformation rules for text-to-text generation, effectively creating "translators" between different writing styles.

Summary updated

Abstract

We address the problem of sentence alignment for monolingual corpora, a phenomenon distinct from alignment in parallel corpora. Aligning large comparable corpora automatically would provide a valuable resource for learning of text-to-text rewriting rules. We incorporate context into the search for an optimal alignment in two complementary ways: learning rules for matching paragraphs using topic structure and further refining the matching through local alignment to find good sentence pairs. Evaluation shows that our alignment method outperforms state-of-the-art systems developed for the same task.
BibTeX
@inproceedings{barzilay2003sentence,
  author = {Regina Barzilay and Noemie Elhadad},
  title = {{Sentence Alignment for Monolingual Comparable Corpora}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2003},
  url = {https://openreview.net/forum?id=B1WKqMfdbH},
}
Figure