TL;DR
This paper introduces LEXVAR, a novel method for automatically building large paraphrase lexicons by exploiting lexical variants (e.g., "amendment" vs. "amending"). By learning generalized paraphrase patterns from seed pairs and applying them to monolingual corpora, LEXVAR significantly expands lexicon coverage while maintaining reasonable quality, outperforming previous methods.
What Problem Does This Paper Address?
The core challenge in building paraphrase lexicons is the trade-off between coverage (how many paraphrases are captured) and accuracy (how correct they are). Existing methods either rely on large, parallel corpora (good for accuracy but limited coverage) or monolingual corpora (high coverage but lower accuracy due to difficulty distinguishing true paraphrases from other semantic relations). This paper aims to bridge this gap by developing a method that leverages the inherent generality of paraphrases, specifically by considering lexical variants, to achieve both high coverage and accuracy from readily available monolingual data.
Methods
The proposed method, LEXVAR, operates in two main steps:
- Learning Paraphrase Patterns: LEXVAR takes a set of high-quality seed paraphrase pairs (SSeed) as input. Unlike previous methods that might generalize only identical word forms, LEXVAR generalizes pairs of lexical variants (e.g., "amendment" and "amending") into more abstract patterns. This is achieved by collecting "affix patterns" that capture these variations (e.g., X:ment of Y ⇔ X:ing Y). These patterns are learned empirically from the seed data.
- Harvesting New Paraphrase Pairs: The learned paraphrase patterns are then used as templates to search through large monolingual corpora. LEXVAR extracts new paraphrase pairs that match these patterns, effectively expanding the lexicon. Crucially, it can discover paraphrases composed of words that are entirely different from the original seed pairs.
The method also includes robust filtering mechanisms for both seed acquisition and harvested pairs to ensure quality.
Datasets
The experiments were conducted using two configurations:
- Europarl setting:
- Bilingual corpus: English-French Europarl Parallel Corpus (2.0M sentence pairs).
- Monolingual corpus: English News Crawl corpora (2011-2013 editions, 52.0M sentences).
- NTCIR setting:
- Bilingual corpus: Japanese-English Patent Translation data (3.2M sentence pairs).
- Monolingual corpus: NTCIR unaligned patent documents (2006-2007 chapters, 39.9M sentences).
For evaluation, news sentences from WMT 2011-2013 "newstest" data (9,000 unique sentences, filtered to 5,850) were used.
Key Insights and Ideas
- Lexical Variants are Key to Generalization: The paper highlights that exploiting the systematic variations between related words (like "amendment" and "amending") is a powerful way to generalize paraphrase patterns beyond simple surface form matching.
- Empirical Pattern Learning: Instead of relying on hand-crafted rules or linguistic theories, LEXVAR learns paraphrase patterns directly from data, making it more adaptable and potentially applicable to various languages.
- Monolingual Corpora are Underrated for Paraphrase Discovery: By combining generalized patterns with large monolingual corpora, the method demonstrates that significant paraphrase expansion is possible without needing vast amounts of parallel data.
- Coverage vs. Accuracy Balance: LEXVAR shows a promising approach to achieving a better balance between lexicon coverage and accuracy, outperforming a baseline that only considers identical word forms.
Key Results
- Significant Expansion: LEXVAR expanded seed sets by factors of 42 to 206 in experiments.
- Increased Coverage: The method achieved 84% to 208% more coverage than a baseline method that only generalizes identical word forms.
- Quality Maintained: Human evaluation demonstrated that the newly acquired paraphrases retained reasonable quality, especially when starting with high-quality seeds.
- LEXVAR output (SLV) is a superset of IDENT output (SID): The proposed method's output always includes the output of the IDENT method, indicating it is a more comprehensive approach.
- Precision: While LEXVAR's expanded lexicon (SLV) showed slightly lower precision than the seed set (SSeed) and the baseline (SID), the scores were still considered reasonably high given the lack of rich language-specific resources.
Limitations and Future Work
- Limited Scope of Lexical Variants: LEXVAR currently only considers prefixes and suffixes. Future work could extend this to other types of affixes (infixes, circumfixes) and even character-level alternations or ideographic languages.
- Corpus Dependency: The method relies on paraphrases appearing in the available corpora. Paraphrases that do not occur in the data cannot be acquired.
- Surface Form Differences: LEXVAR does not exploit paraphrase pairs with completely different surface forms (e.g., "look like" vs. "resemble").
- Future Directions:
- Designing models to harmonize lexicons from different methods and sources.
- Investigating a wider range of corpora and methods for acquiring seed paraphrases (e.g., web-harvested data).
- Applying the method to various languages and extending its handling of lexical variants.
- Evaluating the impact of the expanded paraphrase lexicons on downstream NLP tasks like machine translation and text summarization.
Summary updated