TL;DR
This paper introduces a novel method for automatically acquiring paraphrases by first learning general "paraphrase patterns" from high-quality seed pairs derived from bilingual corpora, and then using these patterns to extract many more paraphrases from abundant monolingual corpora. This approach successfully balances the high precision of bilingual methods with the high recall of monolingual methods, significantly expanding paraphrase collections.
What Problem Does This Paper Address?
Paraphrases, or semantically equivalent expressions in the same language, are crucial for many natural language processing (NLP) tasks. Automatically acquiring large collections of high-quality paraphrases has been a long-standing challenge. Existing methods often face a trade-off: using bilingual parallel corpora yields high-quality but limited paraphrases, while using abundant monolingual corpora offers vast quantities but often suffers from low precision (including non-equivalent pairs like antonyms). This paper aims to bridge this gap by creating a method that leverages both types of corpora to achieve both high quality and high quantity in paraphrase acquisition.
Methods
The proposed method consists of three main steps:
- Seed Paraphrase Acquisition (P<sub>seed</sub>): High-quality seed paraphrases are first obtained from bilingual parallel corpora. The authors utilize a standard phrase-based statistical machine translation (SMT) approach, similar to Bannard and Callison-Burch (2005), to extract candidate pairs. These candidates are then rigorously filtered using techniques like removing statistically unreliable pairs and those consisting solely of stop words, and by comparing phrase probabilities.
- Paraphrase Pattern Induction: General patterns underlying the seed paraphrases are learned. This involves identifying recurring structures and replacing identical or singular-plural word pairs with variable slots (e.g., "X system" ⇒ "X apparatus"). This step essentially generalizes from specific examples to broader rules.
- Paraphrase Instance Acquisition (P<sub>Hust</sub>): The learned paraphrase patterns are then used to harvest new paraphrase instances from large monolingual non-parallel corpora. Expressions that fit the pattern slots are extracted, and their contextual similarity is assessed to ensure they are indeed paraphrases. This step instantiates the general patterns with specific examples found in the monolingual data.
Datasets
The experiments were conducted using two main settings:
- Europarl:
- Bilingual Parallel Corpus: English-French Europarl Parallel Corpus (1.8M sentence pairs).
- Monolingual Data: English side of the Europarl corpus and a 10<sup>9</sup> French-English corpus (23.8M sentences).
- Patent:
- Bilingual Parallel Corpus: Japanese-English Patent Translation data (3.2M sentence pairs).
- Monolingual Data: English side of the patent data and 30.0M sentences from the 2007 chapter of NTCIR unaligned patent documents.
The authors also mention using WMT 2008-2011 "newstest" data for human evaluation.
Key Insights and Ideas
- Generalization and Instantiation: The core idea of learning abstract patterns from high-quality seeds and then applying them to abundant data is a powerful way to scale up NLP resource creation.
- Leveraging Monolingual Data: The paper demonstrates that monolingual corpora, despite their inherent noisiness for paraphrase acquisition, can be effectively mined for high-quality paraphrases when guided by patterns derived from bilingual data.
- Quality vs. Quantity Balance: The method successfully addresses the long-standing trade-off, producing paraphrase collections that are significantly larger than those from bilingual corpora alone, while maintaining a quality comparable to or exceeding existing methods.
- Pattern Rigidity: The paper notes that the "rigid form" of induced patterns can limit coverage (e.g., only 15% of seed paraphrases in Europarl were generalized), suggesting room for more sophisticated pattern induction.
Key Results
- The proposed method significantly expands paraphrase collections. For instance, using the full Patent bilingual corpus, the method acquired 28.7M novel paraphrases (P<sub>Hust</sub>), which is about 21 times the number of seed paraphrases (P<sub>seed</sub>).
- The number of unique LHS phrases covered by P<sub>Hust</sub> was also significantly larger than that of P<sub>seed</sub>.
- The "yield" (average number of RHS phrases per unique LHS phrase) for P<sub>Hust</sub> increased rapidly with the size of the bilingual corpus, outperforming P<sub>seed</sub>, which grew slowly.
- Human evaluation showed that paraphrases from P<sub>Hust</sub> achieved reasonable quality, with average scores for grammaticality and meaning comparable to or better than the best models reported in prior work (e.g., Callison-Burch, 2008). P<sub>seed</sub>-based paraphrases performed even better.
- The study on threshold values (th<sub>p</sub> and th<sub>s</sub>) indicated that better filtering of P<sub>seed</sub> with higher thresholds leads to higher quality P<sub>Hust</sub>, though it may reduce quantity.
Limitations and Future Work
- Coverage of Patterns: The current method's patterns are relatively rigid, leading to a limited coverage of the initial seed paraphrases.
- Quality of P<sub>Hust</sub>: While good, the quality of P<sub>Hust</sub> is dependent on the quality of P<sub>seed</sub>.
- Future Work:
- Investigate more sophisticated similarity metrics for better filtering.
- Explore patterns with more than one variable.
- Conduct learning curve experiments with varying amounts of monolingual data.
- Compare in-domain versus general-purpose monolingual corpora.
- Develop hierarchical (recursive) pattern induction and incorporate lexical resources.
- Apply the developed paraphrase collections to downstream tasks like sentence compression and machine translation.
Summary updated