An effective Discourse Parser that uses Rich Linguistic Information

Rajen Subba, Barbara Di EugenioNAACL 2009

TL;DR

This paper introduces a novel discourse parser that leverages rich linguistic information, including compositional semantics and structural properties, to build hierarchical discourse trees. By employing Inductive Logic Programming (ILP) for relation classification, their approach significantly outperforms traditional machine learning models and baselines, demonstrating the value of deeper linguistic understanding for discourse analysis.

What Problem Does This Paper Address?

Discourse parsing, the task of identifying the hierarchical structure of text and the rhetorical relations between its segments, is a challenging problem in Natural Language Processing (NLP). While previous work has often relied on syntactic and lexical information, this paper argues that a deeper understanding of meaning, specifically compositional semantics, is crucial for accurate discourse parsing. The authors aim to bridge this gap by developing a parser that effectively integrates these richer linguistic features.

Methods

The core of this work lies in two main components:

  • Modified Shift-Reduce Discourse Parser: The authors adapt a standard shift-reduce parsing algorithm. Their modification allows for more flexible attachment of incoming text segments to the emerging tree (at any node on the right frontier) and uses a dedicated relation identification component to determine attachments and relation labels.
  • ILP-Based Rhetorical Relation Classifier: This is the paper's key innovation. Instead of using traditional attribute-value learning algorithms (like Decision Trees or Naive Bayes), they employ Inductive Logic Programming (ILP) with the Aleph system. ILP allows them to learn rules from first-order logic representations, enabling the incorporation of complex linguistic features like compositional semantics and structural information that are difficult to represent in attribute-value formats.

Datasets

  • Discourse Annotated Instructional Corpus: The authors created a corpus of home-repair instructions, manually segmented into Elementary Discourse Units (EDUs). This corpus contains 176 documents, averaging 32.6 EDUs each, for a total of 5744 EDUs and 53,250 words. It is annotated with rhetorical relations and compositional semantics. This corpus differs from existing ones (like the Penn Discourse Treebank) by focusing on instructional texts.

Key Insights and Ideas

  • Compositional Semantics Matters: The paper strongly suggests that going beyond surface-level lexical and syntactic features to incorporate deeper semantic understanding (derived from verb semantics and WordNet) significantly improves discourse relation classification and parsing.
  • ILP's Advantage for Relational Data: Inductive Logic Programming proves to be a powerful tool for discourse parsing because it can naturally handle the relational and structured nature of linguistic information, unlike attribute-value learners.
  • Linguistic Cues are Important but Not Sufficient: While lexical cues and other linguistic signals are useful, the authors note that a significant portion of relations are not explicitly signalled, underscoring the need for inferential capabilities based on deeper meaning.
  • Instructional Texts Have Unique Properties: The study highlights that causal relations are more prevalent in their instructional corpus compared to news articles, suggesting domain-specific characteristics influence discourse structure.

Key Results

  • Relation Classification:
    • The ILP model with semantics achieved an F-Score of 62.78% on Set A (data with full semantic information), significantly outperforming Decision Trees (56.29%), RIPPER (58.02%), and Naive Bayes (35.83%).
  • Even on Set B (all data), the ILP model with semantics (59.43%) outperformed attribute-value models, and crucially, using semantics always improved the ILP model's performance, whereas it sometimes hurt attribute-value models.
  • Discourse Parsing:
    • The modified shift-reduce parser (SR-ILP) using semantics achieved an F-Score of 63.06% on sentence-level relation labeling, significantly outperforming the baseline (34.32%).
  • At the document level, the performance dropped, with the best model achieving 35.44% F-Score for relation labeling, still outperforming the baseline (22.46%).

Limitations and Future Work

  • Data Sparsity: The authors acknowledge that obtaining full compositional semantic data for all EDUs was challenging, leading to the creation of two evaluation sets (Set A and Set B) to assess the impact of semantics.
  • Parsing Complexity: The significant drop in performance at the document level suggests that errors in lower-level attachments can cascade. The authors suggest that a "less greedy approach to parsing discourse structure is warranted."
  • Comparison Challenges: Direct comparison with prior work is difficult due to differences in corpora, relation inventories, and assumptions (e.g., regarding relation directionality).
  • Future Work: The paper implies that further refinement of parsing strategies and potentially more sophisticated ways to handle complex segments (those with multiple nuclei) could lead to improved performance. The domain-independent nature of their ILP model suggests applicability to other text types.

Summary updated

BibTeX
@inproceedings{subba2009effective,
  author = {Rajen Subba and Barbara Di Eugenio},
  title = {{An effective Discourse Parser that uses Rich Linguistic Information}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2009},
}
Figure