TL;DR
This paper proposes a novel approach to automatically parsing the rhetorical structure of instructional text by deeply leveraging event semantics. By moving beyond surface-level features, the authors aim to build more accurate discourse parse trees (DPTs) that capture the underlying informational relationships between text segments, which can benefit downstream NLP applications.
What Problem Does This Paper Address?
Understanding the structure of natural language text, beyond just sentence boundaries, is crucial for many NLP tasks. This structure, often referred to as discourse structure, involves identifying how different parts of a text relate to each other (e.g., cause-effect, elaboration, preparation). Previous work on automatically parsing this structure, known as discourse parsing, has largely relied on syntactic and lexical features, with limited use of deeper semantic information. This approach often struggles with ambiguity and the nuances of how meaning is conveyed. The authors identify a gap in exploiting "deeper semantics," specifically event semantics, to improve the accuracy and robustness of discourse parsing, particularly for instructional texts where understanding the sequence and purpose of actions is key.
Methods
The paper outlines a multi-component system for discourse parsing:
- Discourse Segmenter: A Neural Network model is used to identify Elementary Discourse Units (EDUs), the minimal meaningful segments of text. This model leverages syntactic features and discourse cues and achieves competitive performance.
- Rhetorical Relation Classifier: This is a core novel contribution. Instead of traditional machine learning methods, the authors employ Inductive Logic Programming (ILP) with the Progol system. This allows them to learn rules from rich verb-based event semantic representations of text segments. The background knowledge for Progol consists of these semantic representations, and the positive examples are manually annotated rhetorical relations. This approach allows for learning interpretable rules that capture semantic relationships.
- Discourse Parser: The system builds Discourse Parse Trees (DPTs) in a two-stage process:
- Sentence-Level Parsing: A bottom-up approach combines EDUs into sentence-level DPTs.
- Document-Level Parsing: A shift-reduce parser, adapted from previous work, constructs the overall document structure. This parser uses the rhetorical relation classifier to determine how to combine sentence-level DPTs.
Datasets
- Home Repair Manuals Corpus: The authors used an existing corpus of instructional text for home repair manuals. This corpus was manually annotated with rhetorical relations and event semantic information.
- RST-DT (2002): Used for training and testing the discourse segmenter.
- Preliminary Experiment Data: For the rhetorical relation classifier, a subset of 423 examples was used for training and 85 for testing, focusing on specific relation types (goal:act, step1:step2, before:after, criterion:act) that had at least 50 examples.
Key Insights and Ideas
- Event Semantics as a Powerful Feature: The central idea is that understanding the semantic roles and stages of events described by verbs is crucial for identifying rhetorical relations, going beyond surface-level cues.
- ILP for Interpretable Rule Learning: Using Inductive Logic Programming (ILP) allows the system to learn explicit, human-readable rules for classifying rhetorical relations based on semantic representations. This offers a more transparent approach than black-box machine learning models.
- Compositional Semantic Representations: The integration of VerbNet and CoreLex to build compositional verb-based event semantic representations is a key enabler for the ILP classifier.
- Bridging Sentence and Document Level Parsing: The proposed architecture aims to build a coherent discourse structure from the minimal units (EDUs) up to the entire document level.
Key Results
- Discourse Segmenter: Achieved up to 86.12% F-Score on the RST-DT dataset, comparable to existing state-of-the-art methods.
- Rhetorical Relation Classifier: Outperformed a majority function baseline (51.7% F-Score) with an F-Score of 60.24 on a preliminary experiment. The classifier achieved a total precision of 61.7% and recall of 58.8%.
- Manual Annotation: The manual annotation process yielded a kappa value of 0.66 for inter-annotator agreement on about 26% of the corpus, indicating a reasonable level of consistency.
Limitations and Future Work
- Limited Scope of Preliminary Experiment: The rhetorical relation classifier was tested on a limited set of relations and did not incorporate syntactic or lexical information.
- Future Work:
- Extend the rhetorical relation classifier to incorporate syntactic and lexical information to better assess the contribution of event semantics.
- Build sentence-level and document-level discourse parse trees.
- Investigate how to combine text segments using promotion sets and rank attachment points for sentence-level DPTs.
- Develop a shift-reduce parser for document-level parsing that uses the rhetorical relation classifier and an additional module to determine attachment points.
- Explore the coverage and utility of the lexical resources used.
Summary updated