TL;DR
This paper introduces a novel method for automatically expanding a set of equivalent sentences (paraphrases) by leveraging syntactic substitution. By dynamically acquiring paraphrasing rules through Hierarchical Phrase Alignment and applying them to parse trees, the authors demonstrate a significant quantitative expansion of sentence sets, which is valuable for applications like machine translation evaluation.
What Problem Does This Paper Address?
Generating diverse expressions for the same meaning, known as paraphrasing, is crucial for various natural language processing tasks. A key challenge is creating large, varied sets of equivalent sentences. Existing methods often struggle to generate a wide range of paraphrases systematically. This paper tackles the problem of automatically expanding a given set of equivalent sentences to create a much larger, diverse collection, which is particularly useful for improving the accuracy of machine translation evaluation systems that rely on multiple human-generated reference translations.
Methods
The core of this paper's contribution lies in its two-phase approach:
- Acquisition Phase (Hierarchical Phrase Alignment - HPA): This phase focuses on automatically learning paraphrasing rules. HPA treats two equivalent sentences as a bilingual pair and applies a simplified machine translation approach. It extracts "equivalent phrases" by identifying word correspondences and ensuring that the syntactic categories of the phrase nodes match. This method is designed to capture both semantic and grammatical equivalence, aiming to avoid generating ungrammatical paraphrases. The authors highlight that HPA can extract correspondences even when sentence structures differ significantly.
- Generation Phase (Syntactic Substitution): In this phase, the acquired paraphrasing rules are used to generate new sentences. The process involves selecting a sentence from the source equivalent set, parsing it, and then recursively substituting identified equivalent phrases within its parse tree with their corresponding paraphrases. This structural substitution allows for the generation of phrasal paraphrases, not just lexical ones.
Datasets
The experiments were conducted using 339 source equivalent sentence sets selected from the ATR corpus. These sets were originally created by ten Japanese native speakers translating English sentences into Japanese. The paper notes that, on average, each English sentence had 8.48 equivalent Japanese sentences.
Key Insights and Ideas
- Syntactic Substitution for Paraphrasing: The paper demonstrates that by focusing on structural substitutions within parse trees, one can systematically generate a wide variety of paraphrases, going beyond simple word-level replacements.
- HPA for Robust Rule Acquisition: Hierarchical Phrase Alignment proves effective in extracting semantically and grammatically equivalent phrase pairs, even from sentences with different syntactic structures. This is crucial for ensuring the quality of generated paraphrases.
- Scalability for MT Evaluation: The ability to automatically generate a large number of diverse target sentences from a small initial set has direct implications for improving machine translation evaluation metrics, which benefit from multiple reference translations.
Key Results
The experiments showed that, on average, 561 sentences were correctly generated from an initial set of 8.48 equivalent sentences. This represents an expansion of approximately 66 times. While about 39% of the generated sentences contained errors, the authors argue that the method is effective for creating large sets, as error correction is easier than manual generation. The generated sentences were judged good (grammatically and translationally effective) in approximately 61% of cases.
Limitations and Future Work
The authors acknowledge that a significant portion of generated sentences contain errors (around 39%). Their error analysis indicates that major errors are often caused by inconsistencies in modality and that the method does not explicitly consider pragmatical correctness. Future work includes integrating this method into automatic machine translation evaluation systems.
Summary updated