TL;DR
This paper tackles the challenge of semantic extraction – identifying predicate-argument structures in text – by integrating knowledge from human-built resources like WordNet and FrameNet with Information Extraction (IE) techniques. They propose a novel graphical model that allows for automatic frame assignment, significantly improving recall in semantic role labeling tasks.
What Problem Does This Paper Address?
Natural Language Processing (NLP) systems often struggle with portability and domain independence. A key challenge is effectively harnessing information from large, public knowledge bases like WordNet and FrameNet to build domain-independent NLP solutions. Specifically, this paper focuses on "Semantic Extraction" (SE), which involves assigning semantic bindings to text units (like sentences) to identify predicate-argument structures. While similar to Information Extraction (IE), SE aims for open-domain applicability and is a crucial step in text understanding, unlike IE's typically narrow, pre-specified domains. The authors aim to bridge the gap between existing IE techniques and semantic role labeling approaches by leveraging rich lexical resources.
Methods
The authors combine graphical modeling with Information Extraction (IE) techniques and leverage two key human-built knowledge bases: WordNet and FrameNet.
- FrameNet as Seed Data: They use FrameNet's annotated data to create initial IE patterns and lexicons for specific frame elements.
- WordNet for Lexicon Expansion: To overcome the limited coverage of FrameNet, they use WordNet's hierarchical structure to expand the lexicon. This involves building a graphical model of WordNet and employing a sum-product algorithm for statistical inference to identify relevant WordNet nodes as potential lexical categories for frame elements.
- Statistical Inference: A statistical inference algorithm is used to find relevant WordNet nodes. For frame elements, FrameNet annotations serve as ground truth. The algorithm computes relevance probabilities of higher-level nodes as lexical categories.
- Relevance Metric for WordNet Nodes: To further enhance coverage, they define a relevance metric for WordNet nodes based on their occurrence in IE-related text versus general text. This helps in re-ranking and selecting top relevant nodes to augment the lexicon.
- Graphical Model for Frame Assignment: A graphical model is proposed for the semantic extraction task, integrating automatic frame assignment. This model allows for the estimation of the joint distribution for each FrameNet frame, considering lexical items in a candidate sentence.
Datasets
- FrameNet: Used as a seed dataset for creating initial IE patterns and lexicons.
- WordNet: Used to expand the lexicon for frame elements.
- Yahoo News Service: A corpus of 100 news stories related to "Criminal Investigation" was used for initial experiments. This corpus was also used for evaluating the relevance metric.
Key Insights and Ideas
- Synergy of IE and Semantic Resources: Combining established IE techniques with rich semantic resources like FrameNet and WordNet is a powerful approach for semantic extraction.
- WordNet's Role in Lexicon Expansion: WordNet's hierarchical structure is invaluable for expanding the coverage of frame element lexicons, addressing the limitations of smaller, specialized datasets.
- Graphical Models for Integrated Semantics: The proposed graphical model offers a principled way to integrate frame assignment directly into the semantic extraction process, moving beyond simple role labeling.
- Importance of a Precise Lexicon: Even with limited coverage, a highly precise lexicon and pattern set derived from FrameNet can yield good precision in initial IE experiments.
Key Results
The paper reports improved recall in semantic role assignment tasks:
- Initial Experiment (FrameNet only): Achieved 55% Recall and 68.8% Precision.
- Experiment with WordNet Expansion: After expanding the lexicon with WordNet and applying the relevance metric, the average recall rose to 76.4%, with an average precision of 66%.
Limitations and Future Work
- Limited Coverage: The initial experiments highlighted the limited coverage of FrameNet alone, necessitating the use of WordNet for expansion.
- Ongoing Work on Frame Assignment: The authors acknowledge that their work on automatic frame assignment is ongoing. They are developing a graphical model to estimate the joint distribution for each FrameNet frame and are using a large corpus to estimate frame priors and model parameters. They expect to report more results on this aspect by the time of the conference.
- Assumptions in Graphical Model: For initial experiments, they assume frame elements are conditionally independent given the frame, and that each frame is an independent model. Future work will likely explore relaxing these assumptions.
Summary updated