TL;DR
This paper presents a novel two-level connectionist architecture for resolving structural ambiguity in noun phrases, specifically how prepositional phrases attach. It integrates learned semantic constraints from backpropagation networks with syntactic constraints in a relaxation network, enabling robust disambiguation even for unseen phrases.
What Problem Does This Paper Address?
Natural language processing systems often struggle with structural ambiguity, particularly in how prepositional phrases (PPs) attach within noun phrases. For example, in "the woman wanted the dress on the rack," "on the rack" modifies "dress," while in "the woman positioned the dress on the rack," it modifies "positioned." While previous work focused on verb phrase attachment, this paper tackles PP attachment within noun phrases, which is crucial for understanding complex noun phrases like those found in scientific titles. The challenge lies in effectively combining domain-specific semantic knowledge (e.g., what kind of things can be "in" something) with domain-independent syntactic rules (e.g., locality, avoiding crossing dependencies).
Methods
The authors propose a two-level architecture:
- Bottom Level: Semantic Constraint Learning: This level uses backpropagation networks to learn the plausibility of prepositional relationships between nouns. For each preposition, a separate network is trained on examples of plausible and implausible relationships. Nouns are represented by binary vectors indicating their semantic features (e.g., MEASURING-EVENT, TIME, MATERIAL). The network learns to output a value between 0 and 1 representing plausibility. This approach allows generalization to unseen noun pairs.
- Top Level: Constraint Integration: A relaxation network integrates the learned semantic constraints with syntactic constraints.
- Semantic Nodes: Receive input from the bottom-level backpropagation networks.
- Syntactic Nodes: Represent domain-independent constraints:
- Locality: Favors attachment to closer preceding nouns.
- No-crossing: Prevents attachment structures that cross over each other.
- Structure Nodes: Represent possible structural interpretations of the noun phrase.
The relaxation network allows these constraints to interact, with excitatory connections strengthening plausible interpretations and inhibitory connections weakening competing ones. The network converges to a stable state where the highest-potential structure node represents the preferred interpretation.
Datasets
The experiments used noun phrases extracted from the NPL corpus, which contains titles from scientific and technical domains. For each of the 10 most frequent prepositions, 100 noun phrases were sampled. The authors created training and testing sets for the backpropagation networks, with approximately 200 training examples and 30 testing examples per preposition.
Key Insights and Ideas
- Learned Memory for Semantics: Using backpropagation to learn semantic plausibility for prepositional relationships provides a more powerful and generalizable "memory" than hand-coded rules.
- Two-Level Integration: The separation into a semantic learning level and a syntactic/semantic integration level is effective. The learned semantic knowledge from the bottom level directly informs the top-level relaxation network.
- Semantic Constraints Can Override Syntactic Ones: The paper demonstrates that while syntactic constraints like locality are important, semantic plausibility can sometimes be the deciding factor, even if it means a less local attachment.
- Handling Multiple Prepositions: The architecture is designed to handle noun phrases with multiple prepositional phrases, a significant step beyond previous work that often focused on single PP attachments.
- Generalization: The model can disambiguate noun phrases it has not been explicitly trained on, thanks to the generalization capabilities of both backpropagation and relaxation networks.
Key Results
The backpropagation networks achieved high accuracy in learning prepositional relationships. For example, the network for the preposition "in" correctly classified 93% of training examples and 83% of unseen testing examples. The relaxation network successfully integrated semantic and syntactic constraints to disambiguate noun phrases. In the example "The influence of the radiation intensity on discharges in the Van-Allen-belt," the network correctly identified the preferred attachment structure, demonstrating how semantic plausibility ("influence on discharges" is plausible, "intensity on discharges" is not) can override syntactic locality.
Limitations and Future Work
- Domain Specificity: The semantic features and training data are specific to scientific and technical domains. Extending this to broader domains would require new feature sets and training data.
- Computational Cost: While the approach is connectionist, the training of multiple backpropagation networks and the operation of the relaxation network can be computationally intensive.
- Future Work: The authors suggest their semantic memory model could be integrated into full sentence analyzers for improved sentence processing. They also note that the approach is applicable to other language tasks like word sense disambiguation and compound noun interpretation.
Summary updated