Abstract
This paper demonstrates how a natural language system can learn to identify the antecedents of relative pronouns through an automated approach. By utilizing a conceptual clustering system to create a case-based memory, the system predicts the antecedent of a wh-word based on the preceding clause's description. This automated method matches the performance of laboriously hand-coded rules while requiring only minimal syntactic parsing and general semantic features. Because human intervention is limited to the training phase, this technique allows for the efficient acquisition of disambiguation heuristics tuned to specific domains, offering a generalizable approach for assimilating syntactic and semantic knowledge in natural language processing.
The Problem of Relative Pronoun Disambiguation
In linguistics, a relative pronoun (such as who, which, or that) serves as a link between a main clause and a subsequent description. The noun phrase it refers back to is known as the "antecedent." For example, in the sentence "Tony saw the boy who won the award," the pronoun "who" refers to "the boy."
While humans resolve these links intuitively, they present a significant challenge for automated systems. The antecedent does not occupy a fixed position; it can be the subject, the direct object, or a noun nested within a complex prepositional phrase. Furthermore, antecedents can be single entities, lists (conjunctions), or appositives. Traditionally, resolving these ambiguities required researchers to develop complex, fragile "if-then" rule sets that were difficult to maintain and often failed when encountering new sentence structures. This paper proposes replacing this "knowledge engineering bottleneck" with a system that learns from examples.
Methodology: Case-Based Reasoning and COBWEB
The proposed system treats pronoun disambiguation as a Case-Based Reasoning (CBR) problem. Rather than following rigid logical paths, the system compares a new sentence to a memory of previously encountered examples to find the most similar case.
Representation and Learning
Each "case" in the system's memory represents a sentence containing the pronoun who. These cases are defined by their syntactic structure (e.g., noun phrases, verb phrases) and broad semantic features (e.g., whether a noun represents a human, organization, or physical-target).
The system utilizes COBWEB, a conceptual clustering algorithm, to organize these cases into a hierarchical memory tree. As the system processes training data, it groups similar sentence structures into "probabilistic concepts."
Retrieval and Adaptation
When the system encounters a new sentence (a "probe"), it traverses the COBWEB tree to find the most similar historical case. It then uses the location of the antecedent from that retrieved case to predict the antecedent for the new sentence. If the structures do not match perfectly—for instance, if the retrieved case expects a second noun phrase that the probe lacks—the system applies "adaptation filters" to generalize the result.
Experimental Results
The system was tested using 150 sentences from the MUC-3 corpus, which contains newswire stories regarding Latin American terrorism. The automated approach was compared against two benchmarks: a set of hand-coded rules developed over several months and a "most recent phrase" baseline strategy.
As shown in the results table, the automated system achieved an accuracy between 74% and 89%. Crucially, it matched or outperformed the hand-coded rules in nearly every experiment. When the evaluation was adjusted to discount errors caused by sentence structures entirely absent from the training set, accuracy reached as high as 93%. The system significantly outperformed the baseline strategy, which struggled with the complex linguistic structures common in the MUC-3 dataset.
Insights and Conclusion
The success of this approach highlights several key findings:
- Minimalist Parsing: Deep syntactic analysis is not required; low-level constituents and broad semantic categories are sufficient for high-accuracy disambiguation.
- Domain Tuning: The system automatically adapts to the specific syntactic and semantic quirks of its training data, a task that is difficult to achieve with general-purpose hand-coded rules.
- Reduced Human Effort: Human intervention is shifted from complex programming to the simpler task of labeling antecedents during a training phase.
While this study focused primarily on the pronoun "who," the underlying technique provides a robust framework for acquiring disambiguation heuristics across various natural language tasks, particularly those requiring the integration of syntactic and semantic information.
Summary updated