Unsupervised Induction of Semantic Roles within a Reconstruction-Error Minimization Framework

Ivan Titov, Ehsan Khoddam

TL;DR

This paper introduces a novel unsupervised method for learning semantic roles in sentences by jointly training a feature-rich role labeler and a tensor factorization model. By minimizing reconstruction error of sentence arguments, their approach achieves state-of-the-art results on English and German without relying on language-specific linguistic knowledge, making it more generalizable.

What Problem Does This Paper Address?

Semantic Role Labeling (SRL) aims to identify the predicate-argument structure of sentences, assigning roles like "agent" or "patient" to arguments. Traditional SRL methods are supervised, requiring large amounts of human-annotated data, which is expensive and scarce for many languages and domains. Unsupervised SRL methods, which learn roles from unlabeled text, have historically suffered from strong independence assumptions and simplistic features, limiting their expressiveness and performance, especially for languages with flexible word order. This paper tackles the challenge of learning rich, feature-based semantic role models in an unsupervised manner, bridging the gap between the performance of supervised methods and the data efficiency of unsupervised ones.

Methods

The core of the proposed method is a reconstruction-error minimization framework. It comprises two main components:

  1. Encoding Component (Semantic Role Labeler): This is a feature-rich log-linear model that predicts semantic roles for arguments given the sentence's syntactic and lexical features. This component is designed to be flexible and can incorporate a wide range of features, similar to state-of-the-art supervised SRL systems.
  2. Reconstruction Component (Tensor Factorization Model): This component uses a tensor factorization approach to predict argument fillers based on the assigned semantic roles, the predicate, and other arguments. The idea is that if the induced roles are meaningful, they should help in reconstructing the original sentence's argument structure.

These two components are trained jointly. The objective is to minimize the error in reconstructing sentence arguments. This joint training encourages the semantic role labeler to learn roles that are not only linguistically plausible but also useful for predicting the actual words filling those roles. The authors highlight that this approach is inspired by autoencoders but adapted for learning structured prediction models, allowing for different model families for encoding and reconstruction and focusing reconstruction on specific parts of the input.

Datasets

  • English: The CoNLL 2008 shared task dependency version of PropBank dataset was used. The largest segment (original CoNLL training set, sections 2-21) was used for evaluation and learning, with smaller segments (sections 22-23) for model development.
  • German: The CoNLL 2009 version of the SALSA corpus was used. The authors note that they re-parsed this corpus with a dependency parser and used a supervised classifier for argument identification, mirroring the setup in Titov and Klementiev [52].

The paper does not introduce new benchmarks but replicates experimental setups from previous work to allow for direct comparison.

Key Insights and Ideas

  • Reconstruction Error as a Driving Force for Unsupervised Learning: The paper demonstrates that optimizing for the accurate reconstruction of sentence arguments, given predicted roles, is a powerful objective for inducing meaningful semantic roles without supervision. This is a departure from traditional generative or clustering-based unsupervised SRL methods.
  • Bridging the Gap Between Supervised and Unsupervised SRL: By using a feature-rich encoder similar to supervised models, the method can capture complex syntax-semantics interactions, overcoming a major limitation of previous unsupervised approaches.
  • Language Independence: The proposed method achieves competitive results on both English and German without incorporating any language-specific linguistic knowledge or argument signatures, suggesting a more generalizable approach to unsupervised SRL.
  • Flexibility in Model Choice: The framework allows for virtually any computationally tractable structured model to be used as the role labeler, opening doors for integrating existing SRL systems or exploring new ones.
  • Potential for Semi-Supervised Learning: The authors suggest that the reconstruction objective can be easily combined with likelihood objectives, paving the way for powerful semi-supervised SRL methods.

Key Results

  • State-of-the-Art Performance: The proposed model achieves results on par with or outperforming the most accurate previous unsupervised semantic role induction methods on both English and German datasets, as measured by F1 scores.
  • English Results: Achieved an F1 score of 82.8%, comparable to the best baseline (Bayes at 82.5%). Notably, the model induces fewer roles (4-6) than some baselines, leading to higher collocation scores, suggesting more interpretable roles.
  • German Results: Achieved an F1 score of 81.3%, competitive with specialized models like Bayes+LangSpecific (80.9%). Again, the model's ability to induce fewer, more coherent roles is highlighted.
  • Feature Richness Advantage: The paper emphasizes that their feature-rich model, without language-specific priors, outperforms methods that rely on such specialized features (e.g., the drop in performance for a Bayesian model when using English-specific signatures on German).

Limitations and Future Work

  • Role Interpretability: While the model achieves good performance, the induced roles are not human-interpretable by default (as is common in unsupervised latent variable models). The paper notes that their model induces fewer roles, which aids interpretability compared to methods that might induce many fine-grained roles.
  • Verb Sense Induction: The authors observed that in some regimes, their model starts to induce roles specific to individual verb senses or groups of semantically similar predicates. They suggest that explicitly incorporating a latent variable for predicate senses could lead to more informative semantic representations and improve role induction performance. This is identified as a direction for future work.
  • Semi-Supervised Extension: The authors propose that their reconstruction objective can be combined with likelihood objectives to create powerful semi-supervised methods, which they leave for future exploration.

Summary updated

Abstract

We introduce a new approach to unsupervised estimation of feature-rich semantic role labeling models. Our model consists of two components: (1) an encoding component: a semantic role labeling model which predicts roles given a rich set of syntactic and lexical features; (2) a reconstruction component: a tensor factorization model which relies on roles to predict argument fillers. When the components are estimated jointly to minimize errors in argument reconstruction, the induced roles largely correspond to roles defined in annotated resources. Our method performs on par with most accurate role induction methods on English and German, even though, unlike these previous approaches, we do not incorporate any prior linguistic knowledge about the languages.
BibTeX
@inproceedings{titov2015unsupervised,
  author = {Ivan Titov and Ehsan Khoddam},
  title = {{Unsupervised Induction of Semantic Roles within a Reconstruction-Error Minimization Framework}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2015},
  url = {https://openreview.net/forum?id=Hk-187-_bB},
}

Loading related content...

Figure