TL;DR
This paper presents a novel method for automatically discovering relationships between words (arguments) and the actions or states they are associated with (predicates), even when the predicate is implied rather than explicitly stated. By leveraging a semantically annotated corpus and a larger unannotated corpus for ranking, they achieve a significant improvement over random chance in identifying these implicit relationships, paving the way for richer semantic understanding in NLP.
What Problem Does This Paper Address?
Understanding the implicit relationships between words is crucial for natural language processing (NLP). For instance, the word "cigarette" strongly implies the action of "smoking." This phenomenon, known as logical metonymy and other forms of implicit predicate inference, means that a full understanding of a sentence often requires inferring predicates that aren't explicitly present. While this is widely accepted in linguistics, there's a lack of comprehensive lexical semantic resources that capture these implicit relationships. Existing approaches either rely on manual creation, which is time-consuming and expensive, or automatic methods that struggle to capture the nuances of semantic relatedness. This paper aims to bridge this gap by automatically acquiring these argument-predicate relations from a specialized corpus.
Methods
The core of the proposed method relies on two key assumptions:
- Predictability: If a predicate is highly predictable from an argument's semantics, it can be omitted in discourse.
- Frequency: If a predicate frequently takes a word as an argument, it's highly predictable from that word's semantics.
The authors leverage the SALSA corpus, a German newspaper corpus annotated with FrameNet frames. FrameNet provides a structured way to represent semantic roles (like "Agent," "Patient") within "frames" (representing situations or events). The approach involves:
- Identifying Candidate Relations: For each content word in SALSA, they identify potential predicates that frequently co-occur with it.
- Defining Argument-Predicate Pairs: They then define argument-predicate relations by mapping the identified arguments to specific roles within FrameNet frames evoked by the predicates.
- Estimating Semantic Relatedness: Crucially, SALSA is too small to reliably estimate the semantic relatedness of these extracted pairs. To overcome this, they use a much larger, unannotated German newspaper corpus (Süddeutsche Zeitung, 145 million words) for this purpose. They employ Latent Semantic Analysis (LSA) to create a semantic space and calculate word similarity. The final relatedness score combines the LSA score with the frequency of the relation in SALSA (normalized by confidence scores).
Datasets
- SALSA Corpus: A German newspaper corpus manually annotated with FrameNet frames and role-semantic information. Used for extracting candidate argument-predicate relations.
- Süddeutsche Zeitung (SZ) Corpus: A large, unannotated German newspaper corpus (145 million words). Used for computing semantic relatedness via LSA.
- Gold Standard: A set of 30 test arguments for which human subjects provided associated predicates. This was used for evaluation.
Key Insights and Ideas
- Leveraging Annotated Corpora for Implicit Relations: The paper demonstrates that even a moderately sized, manually annotated corpus like SALSA can be a valuable resource for extracting implicit semantic relations, provided it's combined with a larger corpus for robust relatedness estimation.
- FrameNet as a Unifying Framework: Using FrameNet frames allows for a fine-grained and grounded representation of argument-predicate relations, going beyond simple noun-verb pairs and accommodating arguments of various parts of speech.
- The Power of LSA for Relatedness: LSA proves effective in estimating semantic relatedness, especially when the primary corpus (SALSA) is too small for reliable co-occurrence statistics.
- Human Agreement is Challenging: The experiment to create the gold standard revealed low inter-annotator agreement among human subjects, highlighting the subjective nature of inferring implicit predicates and the difficulty of creating consistent gold standards for such tasks.
Key Results
- Improved Ranking: The proposed method, using semantic relatedness ranking, significantly outperformed random ranking in evaluating against the gold standard. The maximal F-measure (Fmax) achieved was 0.47 for the semantic-based procedure, compared to a baseline of 0.18.
- Semantic vs. Syntactic Extraction: The procedure using semantic annotation (FrameNet frames) performed better (Fmax 0.47) than one relying solely on syntactic annotation (Fmax 0.41), suggesting the value of semantic information.
- Correlation with Human Judgment: The automatically calculated relatedness measure showed a statistically significant correlation (Spearman's rho = 0.30, p=0.0006) with human judgments of naturalness, indicating that the system's ranked relations align with human intuition.
Limitations and Future Work
- Corpus Size: The SALSA corpus, despite being annotated, is still relatively small, limiting the number of frequently occurring relations.
- Anaphora Resolution: The current anaphora resolution is basic (pronoun-to-noun matching) and could be improved with more advanced techniques to increase precision.
- Compound Splitting: German compounds are treated as single lexemes; splitting them could reveal additional semantic evidence.
- Human Agreement: The low human agreement in the gold standard task needs further investigation.
- Future Work: The authors plan to:
- Implement more advanced anaphora resolution.
- Explore larger, role-annotated corpora (like PropBank and FrameNet for English) which will require parsing.
- Investigate the relationship between human agreement and semantic types of arguments.
- Develop procedures to automatically extend phrases containing implicit predicates and compare these with human responses.
- Explore other corpus-based relatedness measures, including web-based ones.
Summary updated