UC Davis at SemEval-2019 Task 1: DAG Semantic Parsing with Attention-based Decoder

Dian Yu, Dian Yu 0002, Kenji Sagae

Abstract

Semantic parsing into Directed Acyclic Graphs (DAGs) presents unique challenges, such as handling reentrancy and discontinuous structures. Researchers from UC Davis propose a simplified encoder-decoder model for the Universal Conceptual Cognitive Annotation (UCCA) framework. By utilizing a recursive attention-based decoder instead of complex transition-based states or heavy graph-scoring mechanisms, the model achieves competitive results on SemEval-2019 Task 1, particularly in identifying primary semantic relationships.

Introduction

Traditional syntactic parsing typically results in tree structures. However, capturing the full semantic meaning of a sentence often requires more complex representations. The Universal Conceptual Cognitive Annotation (UCCA) framework addresses this by representing meaning as a Directed Acyclic Graph (DAG).

UCCA introduces two primary challenges for standard parsers:

  1. Reentrancy: A single concept may fulfill multiple semantic roles, represented as a node with multiple parents in the graph.
  2. Discontinuity: Semantic units can be split across a sentence (e.g., phrasal verbs like "turned it down").

While existing models often rely on complex transition-based states or exhaustive graph-scoring, this work explores whether a simpler, attention-driven architecture can effectively perform semantic parsing without explicit state tracking.

Methodology

The proposed model follows an encoder-decoder architecture designed for simplicity and efficiency.

Encoder

The encoder uses a bidirectional LSTM (BiLSTM) to process input sentences. To mitigate the limitations of a small dataset, the input is enriched by concatenating fastText word embeddings with part-of-speech (POS) tags, entity types, and casing information.

Recursive Attention Decoder

Instead of a traditional RNN, the decoder employs a recursive attention mechanism to build the DAG. It identifies the boundaries of semantic spans and uses a biaffine-inspired multiplication model to score potential connections between nodes. The process is recursive, continuing until a node "attends to itself," signaling the completion of a span.

Node and Span Representations

Terminal nodes correspond to individual words. Non-terminal nodes, representing phrases or concepts, are modeled using span embeddings—specifically, the vector difference between the span's boundary representations. Specialized multi-layer perceptrons (MLPs) are further used to classify proper nouns and identify discontinuous units.

Example of a UCCA Directed Acyclic Graph (DAG) for the sentence 'Mariah Carey turned it down,' illustrating reentrancy and discontinuity.

Experimental Setup

The model was evaluated on the English Wiki corpus as part of the SemEval-2019 Task 1 "closed track." The dataset is relatively small, consisting of 4,113 training sentences, 514 for development, and 515 for testing. The authors utilized standard NLP tools like spaCy for preprocessing and did not use external datasets beyond pretrained embeddings.

Results

The model demonstrated strong performance, ranking third in the in-domain English task. It proved particularly effective at identifying primary semantic relationships.

Performance comparison on primary and remote edges for baseline, official, and final model versions.

As shown in the results, the final model achieved a macro-averaged F1-score of 0.864 for labeled primary edges and 0.753 for unlabeled primary edges. The recursive attention approach successfully captures the core semantic structure even with limited training data.

Discussion and Conclusion

Despite its success with primary edges, the model faced challenges with "remote edges"—long-distance semantic dependencies that are sparse in the training data. Performance on discontinuous units was also limited by the small size of the Wiki corpus.

Future work may involve refining the attention mechanism or incorporating more advanced pre-trained language models like BERT to improve the detection of remote dependencies and enhance overall robustness.

Summary updated

Abstract

We present an encoder-decoder model for semantic parsing with UCCA SemEval 2019 Task 1. The encoder is a Bi-LSTM and the decoder uses recursive self-attention. The proposed model alleviates challenges and feature engineering in traditional transition-based and graph-based parsers. The resulting parser is simple and proved to effective on the semantic parsing task.
BibTeX
@inproceedings{yu2019uc,
  author = {Dian Yu and Dian Yu and Kenji Sagae},
  title = {{UC Davis at SemEval-2019 Task 1: DAG Semantic Parsing with Attention-based Decoder}},
  booktitle = {SemEval @ NAACL},
  year = {2019},
  doi = {10.18653/v1/s19-2017},
}
Figure