Understanding the semantic relationship between a noun and its modifier is a fundamental challenge in Natural Language Processing. For instance, "iron gate" implies a MATERIAL relation, while "flu virus" suggests a CAUSAL one. This paper investigates the effectiveness of two distinct feature sets—structured lexical hierarchies (WordNet) and corpus-based grammatical patterns (Word Sketches)—in classifying these relationships.
Problem Statement
Noun-modifier pairs are often ambiguous. Unlike verbs, which have well-defined thematic roles for their arguments, the semantic link in phrases like "desert storm" (Spatial) or "weekly game" (Temporal) must be inferred from the context or inherent properties of the words. The authors aim to determine whether hand-coded semantic knowledge or observed linguistic behavior provides a better foundation for supervised machine learning models to categorize these relations across five broad classes: Causal, Participant, Spatial, Temporal, and Quality.
Methodology
The study evaluates two primary representation methods:
- WordNet-based Representation: This approach leverages the WordNet hierarchy by extracting hypernyms (ancestors) for each word in a pair. It was tested in two scenarios: one with gold-standard word-sense disambiguation (WSD) and one where all possible senses were aggregated.
- Corpus-based Representation (Word Sketches): Using the British National Corpus, the authors generated "Word Sketches"—summaries of a word’s grammatical behavior, such as its typical subjects, objects, and modifiers. These collocations act as a proxy for semantic meaning without requiring manual sense tagging.
These features were processed using three machine learning algorithms: TiMBL (instance-based), C5.0 (decision trees), and SVMlight (Support Vector Machines).
Experimental Setup
The experiments utilized a dataset of 600 noun-modifier pairs. Performance was measured using Precision, Recall, and F-score, with comparisons against established benchmarks like Latent Relational Analysis (LRA).
Key Insights and Results
The results highlight a significant "sense gap": WordNet-based models perform exceptionally well when provided with precise word-sense information but degrade significantly without it. In contrast, corpus-based "Word Sketches" proved more robust in the absence of manual sense tagging.
- Temporal Relations: Consistently the easiest to classify, with F-scores reaching approximately 82%. This is likely due to the modifiers belonging to a distinct, small set of time-related concepts.
- Robustness of Word Sketches: For the "Participant" class, Word Sketches achieved an F-score of ~54%, significantly outperforming WordNet when senses were unknown (~33%).
- Feature Selection: Decision trees (C5.0) identified specific "smoking gun" features, such as the proximity of the word "happen" to a modifier, as strong indicators of Temporal relations.
Conclusion and Future Directions
While WordNet with sense information remains the "gold standard," the high cost of manual disambiguation makes corpus-based features a compelling, scalable alternative. The authors suggest that future work could utilize co-training—where models trained on different feature sets "teach" each other—to automatically label larger datasets and overcome data sparseness issues.
Summary updated