TL;DR
This paper introduces novel unsupervised methods for resolving structural ambiguity in natural language by leveraging the vastness of the web as an implicit training set. By cleverly querying search engines for surface features and paraphrases, they achieve competitive results on challenging tasks like prepositional phrase attachment and noun compound coordination without relying on manually labeled data.
What Problem Does This Paper Address?
Natural language processing (NLP) tasks often struggle with "structural ambiguity," where a sentence can have multiple valid grammatical interpretations. A prime example is prepositional phrase (PP) attachment: in "I saw the man with the telescope," does the telescope belong to "me" (the saw) or the "man"? Resolving these ambiguities is crucial for accurate parsing and understanding. Traditionally, this requires large amounts of hand-labeled data, which is expensive and time-consuming to create. Furthermore, even with labeled data, the "data sparseness" problem arises – many word combinations are rare, making it hard to gather sufficient statistics. This paper aims to overcome these limitations by using the web as a massive, readily available source of implicit training data.
Methods
The authors propose two key unsupervised techniques to extract implicit labels from the web:
- Web-Derived Surface Features: They exploit the idea that writers often use subtle surface-level cues (like punctuation, capitalization, or specific phrasing within parentheses) to signal the intended meaning. By querying search engines and analyzing the results, they look for these markers. For instance, "open the door (with a key)" suggests the PP "with a key" attaches to the verb "open."
- Paraphrases: This method involves rephrasing the ambiguous structure in different ways and checking if these paraphrases appear frequently in web search results. If a paraphrased version that implies a specific attachment (e.g., noun attachment vs. verb attachment) is common, it suggests that this is the intended interpretation. They define several patterns for paraphrasing PP attachment and noun compound coordination.
These techniques are applied to n-gram models (calculating probabilities of word co-occurrences) to infer attachment decisions.
Datasets
The primary dataset used for evaluation is the benchmark dataset created by Ratnaparkhi et al. (1994) for PP-attachment, consisting of 3,097 examples. For noun compound coordination, they use a collection of 428 examples extracted from the Penn Treebank. The web itself serves as the implicit training corpus.
Key Insights and Ideas
- The Web as a "Free" Labeled Corpus: The core idea is that the sheer volume of text on the web, combined with search engine capabilities, can act as a massive, implicitly labeled dataset, bypassing the need for manual annotation.
- Surface Cues Carry Semantic Weight: Writers often unconsciously embed disambiguating information in the surface form of their text. Exploiting these subtle cues can be surprisingly effective.
- Paraphrasing as a Disambiguation Tool: The ability to find frequent paraphrases of an ambiguous structure provides strong evidence for a particular interpretation.
- Unsupervised Methods Can Compete: The paper demonstrates that unsupervised approaches, when leveraging large web corpora, can achieve performance comparable to or even exceeding supervised methods, without the associated annotation costs.
Key Results
The paper reports strong performance on two key structural ambiguity tasks:
- Prepositional Phrase (PP) Attachment: They achieve 84% precision on PP-attachment tasks.
- Noun Compound Coordination: They achieve 80% precision on noun compound coordination.
These results are competitive with state-of-the-art supervised methods, and in some cases, surpass them, particularly when considering the unsupervised nature of their approach.
Limitations and Future Work
- Test Set Errors: The authors note that the PP-attachment test set contains errors, which can affect unsupervised algorithms that cannot learn to compensate for them.
- Search Engine Limitations: Search engines sometimes ignore punctuation, which can hinder the extraction of certain surface features.
- Future Work: The authors plan to explore more sophisticated "evidence combination algorithms" and apply their approach to other NLP problems. They also aim to investigate better-motivated ways to combine evidence from different sources.
Summary updated