DeepCx: A transition-based approach for shallow semantic parsing with complex constructional triggers

Jesse Dunietz, Jaime G. Carbonell, Lori S. LevinEMNLP 2018 · DOI

Introduction

Traditional Shallow Semantic Parsing (SSP) frameworks, such as PropBank and FrameNet, typically operate under a "word-centric" assumption. They presume that semantic relations are triggered by single lexical units or simple multi-word expressions. However, natural language frequently utilizes constructions—complex, often discontinuous patterns where the trigger is distributed across a sentence (e.g., "so [X] that [Y]").

To address this, the authors propose Surface Construction Labeling (SCL), a task that treats these complex patterns as primary semantic triggers. They introduce DeepCx, a neural transition-based parser specifically designed to handle the "gappy" and overlapping nature of constructional triggers.

The DeepCx Transition System

DeepCx employs a state-machine approach inspired by dependency parsing. It processes a sentence through a sequence of transitions to construct a semantic map. Key transitions include:

  • NEW-CONN and CONN-FRAG: These allow the system to identify multi-word or discontinuous connectives.
  • LEFT-ARC and RIGHT-ARC: These link words to specific semantic roles (e.g., Cause or Effect) relative to a connective anchor.
  • SPLIT: A specialized action for handling overlapping constructions where a single word participates in multiple relations.

Neural Architecture

The parser's state is represented by a high-dimensional vector that integrates multiple sources of information to predict the next transition in the sequence.

DeepCx Neural Architecture for State Representation

As shown in the architecture diagram, the state vector is composed of several key components:

  • Stack LSTMs: Representing the buffer of unprocessed words and the stack of processed words.
  • Span Embeddings: Tracking the content of connective, cause, and effect spans as they are constructed.
  • Dependency Paths: Capturing syntactic relationships between triggers and arguments.
  • Action History: Encoding the sequence of previous transitions.

Evaluation on Causal Language

DeepCx was evaluated using the BECAUSE 2.1 Corpus, which contains approximately 4,800 sentences annotated for causal connectives, causes, effects, and means. This dataset is particularly challenging due to its inclusion of complex and discontinuous causal constructions.

Results and Discussion

DeepCx demonstrates a significant performance improvement over Causeway, the previous state-of-the-art construction-based parser.

Performance Comparison of DeepCx against Baselines and Ablations

The results indicate that DeepCx achieves substantial gains, particularly in recall. For instance, the Vanilla DeepCx model improves the Connective $F_1$ from 53.1 to 59.2 and the Effect Argument $F_1$ from 30.4 to 41.0. Ablation studies reveal that while dependency paths and action history contribute to performance, the neural model's ability to capture sequence information implicitly via LSTMs is a primary driver of its success. Interestingly, explicit dependency paths provided only marginal improvements, suggesting the model captures much of this structural information implicitly.

DeepCx also shows strong generalization capabilities, successfully identifying "novel" connectives not seen during training by leveraging the surrounding semantic and syntactic context.

Limitations and Future Work

Despite its success, DeepCx has several limitations:

  • Sub-lexical Triggers: It cannot currently handle triggers embedded within words (e.g., the comparative suffix "-er").
  • Implicit Relations: The system requires an explicit lexical trigger and cannot identify relations implied solely by juxtaposition.
  • Scope: The current model is limited to sentence-level parsing and does not account for cross-sentential relations.
  • Data Availability: Expanding SCL to other semantic domains (e.g., concession or comparison) requires the development of new specialized datasets.

Summary updated

BibTeX
@inproceedings{dunietz2018deepcx,
  author = {Jesse Dunietz and Jaime G. Carbonell and Lori S. Levin},
  title = {{DeepCx: A transition-based approach for shallow semantic parsing with complex constructional triggers}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1196},
}
Figure