Abstract
We address the problem of sentence alignment for monolingual corpora, a phenomenon distinct from alignment in parallel corpora. Aligning large comparable corpora automatically would provide a valuable resource for learning of text-to-text rewriting rules. We incorporate context into the search for an optimal alignment in two complementary ways: learning rules for matching paragraphs using topic structure and further refining the matching through local alignment to find good sentence pairs. Evaluation shows that our alignment method outperforms state-of-the-art systems developed for the same task.
Introduction
In Natural Language Processing (NLP), parallel corpora—direct translations of the same text—have long been the gold standard for training translation models. However, "comparable corpora," which consist of texts in the same language describing the same events or concepts in different styles (e.g., expert vs. layperson), are far more abundant. These corpora are essential for tasks like text simplification and summarization.
The primary challenge lies in the "noisy" nature of comparable corpora. Unlike translations, the information order can vary significantly, and many sentences may lack a direct equivalent. Traditional alignment tools often fail because they assume linear progression or rely on simple lexical overlap, which cannot distinguish between true paraphrases and unrelated sentences sharing common words.
Proposed Methodology
The researchers propose a hierarchical approach that leverages the topical structure of documents to guide sentence alignment. This method moves beyond simple word-matching by considering the context in which a sentence appears.
1. Topical Structure Induction
The system first identifies recurring "topics" at the paragraph level across the entire collection. Using hierarchical clustering, it groups paragraphs that convey similar types of information, such as "history" or "geography" in city descriptions.
2. Macro Alignment
Using a small training set, the system learns mapping rules between these clusters. For instance, it might learn that a "symptoms" paragraph in a technical medical text typically corresponds to a "symptoms" paragraph in a simplified version.
3. Micro Alignment
By mapping paragraphs first, the search space for sentence alignment is drastically reduced. The system then employs a modified version of the Needleman-Wunsch algorithm to find the best sentence-level matches within those mapped paragraphs, allowing for many-to-many alignments.
Experimental Setup
The method was evaluated using a dataset compiled from Encyclopedia Britannica and Britannica Elementary.
- Data: 103 pairs of city descriptions (e.g., "Lisbon," "Jakarta").
- Characteristics: The elementary version is a simplified subset but often contains updated or distinct details.
- Split: 92 pairs were used for training/clustering, and 11 pairs were reserved for a human-annotated test set.
Results and Discussion
The evaluation demonstrates that context is a more reliable indicator of a match than lexical similarity alone. A simple cosine similarity measure, when constrained by topical context, outperformed sophisticated systems like SimFinder.
At a fixed recall level of 55.8%, the "Full Method" achieved a precision of 76.9%, significantly higher than SimFinder (24.0%) and the cosine baseline (57.9%). When provided with "ideal" manual clusters, precision reached 80.9%, indicating that while the unsupervised clustering is effective, there is still room for improvement.
Conclusion
This hierarchical alignment method exploits the inherent topical regularity of specific domains. While it excels in structured environments like encyclopedias, its performance in highly heterogeneous collections remains an area for future research. Ultimately, this approach paves the way for automatically inducing transformation rules for text-to-text generation, effectively creating "translators" between different writing styles.
Summary updated