Unsupervised Learning of Prototypical Fillers for Implicit Semantic Role Labeling

Niko Schenk, Christian ChiarcosNAACL 2016

TL;DR

This paper introduces a novel unsupervised approach to identify "implicit" semantic roles in text, which are roles that are understood but not explicitly stated. By learning "prototypical fillers" from existing semantic role labeling data, the method can identify these missing roles without requiring any new manual annotations, achieving competitive performance with supervised methods.

What Problem Does This Paper Address?

Semantic Role Labeling (SRL) aims to identify predicates (verbs or nouns) in text and their associated arguments (who did what to whom, where, when, etc.). However, a significant challenge in SRL is identifying "implicit" or "null instantiated" roles (NIs). These are roles that are semantically present but not explicitly expressed in the sentence, often requiring understanding context from surrounding sentences. Traditional supervised SRL methods struggle with NIs because manually annotating them is extremely costly, time-consuming, and results in very sparse datasets. Existing approaches to implicit SRL (iSRL) are also supervised and suffer from this data scarcity. This paper tackles the problem of iSRL by developing a method that bypasses the need for manually annotated iSRL data.

Methods

The core of the proposed method lies in learning "prototypical fillers" for each semantic role. This is achieved by:

  1. Inducing Prototypical Fillers: The authors leverage large amounts of explicit SRL annotations (from resources like FrameNet and PropBank/NomBank) to learn representative word embeddings for words that fill specific roles. For a given role, a "protofiller" is computed as the average of the embeddings of all words that have appeared in that role.
  2. Identifying Null Instantiations: Once protofillers are learned, the system searches for candidate constituents (nouns, verbs, PPs) in the surrounding text. These candidates are then compared to the protofiller of a target role using cosine similarity. The candidate that maximizes this similarity is predicted as the null instantiation for that role. The method uses word embeddings (like Collobert-Weston and word2vec) to represent words and their semantic relationships.

This approach is unsupervised in the sense that it doesn't require new manual annotations specifically for implicit roles. It relies on existing explicit SRL annotations and pre-trained word embeddings.

Datasets

The authors used the following datasets:

  • Training Data:
    • CLMET (Corpus of Late Modern English Texts): Annotated with FrameNet-style semantic roles using SEMAFOR.
  • Gigaword Corpus: Annotated with PropBank/NomBank-style roles using MATE.
  • Evaluation Data:
    • SemEval 2010 Shared Task data: A collection of fiction novels with manual annotations of null instantiations.
  • Gerber and Chai (2010)'s augmented NomBank data: Focused on nominal predicates.

Key Insights and Ideas

  • Leveraging Explicit Annotations for Implicit Tasks: The paper demonstrates that existing, readily available explicit SRL annotations can be effectively repurposed to tackle the challenging iSRL problem, significantly reducing the annotation burden.
  • "Protofillers" as Semantic Signatures: The concept of "prototypical fillers" acts as a powerful semantic signature for each role. By averaging embeddings, the method captures the essence of what typically fills a role, allowing it to generalize beyond specific instances.
  • Distributional Similarity for Role Resolution: The core idea of using distributional similarity (via word embeddings) to match candidate phrases with role prototypes is a key insight. This allows the system to find semantically plausible fillers even when they are not explicitly marked.
  • Contextual Window is Crucial: The effectiveness of the method is shown to be sensitive to the context window used for identifying candidate NIs, suggesting that discourse-level information is vital for iSRL.

Key Results

  • Competitive Performance on SemEval: The proposed method achieved competitive performance on the SemEval 2010 iSRL task, outperforming a similar vector-based resolver (Gorinski et al., 2013) and being comparable to state-of-the-art supervised methods (Silberer and Frank, 2012) in terms of recall.
  • Strong Performance on NomBank: On the NomBank dataset, the method also showed strong results, outperforming previous baselines.
  • Effectiveness of Embeddings: The choice of word embeddings significantly impacts performance, with Collobert et al. (2011) embeddings and custom-trained continuous bag-of-words embeddings showing the best results. Dependency-based embeddings and skip-gram embeddings performed worse.
  • Contextual Accuracy: The method was particularly effective for NIs encountered within the same sentence as the target predicate (44.4% accuracy).

Limitations and Future Work

  • Performance Gap with Supervised Methods: While competitive, the method does not consistently outperform the best supervised methods, particularly in terms of F1 score on the SemEval task.
  • Nominal Predicates Challenge: Inferring implicit roles for nominal predicates was found to be more challenging due to greater variation in fillers (e.g., protoagents of "loan" could be institutions or countries), leading to noise.
  • Future Work Suggestions:
    • Protofiller Clusters: The authors suggest exploring "topic-like" protofiller clusters as a promising alternative to handle the variation in fillers for nominal predicates.
  • Cross-lingual Adaptation: The approach is seen as a promising technique for cross-lingual iSRL, especially with the increasing availability of Universal Dependencies and efforts to harmonize syntax annotations across languages.
  • Broader Role Inventories: The method is not restricted to FrameNet or PropBank roles and can be applied to other semantic role inventories, including syntactic dependency annotations.

Summary updated

BibTeX
@inproceedings{schenk2016unsupervised,
  author = {Niko Schenk and Christian Chiarcos},
  title = {{Unsupervised Learning of Prototypical Fillers for Implicit Semantic Role Labeling}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2016},
}
Figure