Abstract
Semantic parsing into Directed Acyclic Graphs (DAGs) presents unique challenges, such as handling reentrancy and discontinuous structures. Researchers from UC Davis propose a simplified encoder-decoder model for the Universal Conceptual Cognitive Annotation (UCCA) framework. By utilizing a recursive attention-based decoder instead of complex transition-based states or heavy graph-scoring mechanisms, the model achieves competitive results on SemEval-2019 Task 1, particularly in identifying primary semantic relationships.
Introduction
Traditional syntactic parsing typically results in tree structures. However, capturing the full semantic meaning of a sentence often requires more complex representations. The Universal Conceptual Cognitive Annotation (UCCA) framework addresses this by representing meaning as a Directed Acyclic Graph (DAG).
UCCA introduces two primary challenges for standard parsers:
- Reentrancy: A single concept may fulfill multiple semantic roles, represented as a node with multiple parents in the graph.
- Discontinuity: Semantic units can be split across a sentence (e.g., phrasal verbs like "turned it down").
While existing models often rely on complex transition-based states or exhaustive graph-scoring, this work explores whether a simpler, attention-driven architecture can effectively perform semantic parsing without explicit state tracking.
Methodology
The proposed model follows an encoder-decoder architecture designed for simplicity and efficiency.
Encoder
The encoder uses a bidirectional LSTM (BiLSTM) to process input sentences. To mitigate the limitations of a small dataset, the input is enriched by concatenating fastText word embeddings with part-of-speech (POS) tags, entity types, and casing information.
Recursive Attention Decoder
Instead of a traditional RNN, the decoder employs a recursive attention mechanism to build the DAG. It identifies the boundaries of semantic spans and uses a biaffine-inspired multiplication model to score potential connections between nodes. The process is recursive, continuing until a node "attends to itself," signaling the completion of a span.
Node and Span Representations
Terminal nodes correspond to individual words. Non-terminal nodes, representing phrases or concepts, are modeled using span embeddings—specifically, the vector difference between the span's boundary representations. Specialized multi-layer perceptrons (MLPs) are further used to classify proper nouns and identify discontinuous units.
Experimental Setup
The model was evaluated on the English Wiki corpus as part of the SemEval-2019 Task 1 "closed track." The dataset is relatively small, consisting of 4,113 training sentences, 514 for development, and 515 for testing. The authors utilized standard NLP tools like spaCy for preprocessing and did not use external datasets beyond pretrained embeddings.
Results
The model demonstrated strong performance, ranking third in the in-domain English task. It proved particularly effective at identifying primary semantic relationships.
As shown in the results, the final model achieved a macro-averaged F1-score of 0.864 for labeled primary edges and 0.753 for unlabeled primary edges. The recursive attention approach successfully captures the core semantic structure even with limited training data.
Discussion and Conclusion
Despite its success with primary edges, the model faced challenges with "remote edges"—long-distance semantic dependencies that are sparse in the training data. Performance on discontinuous units was also limited by the small size of the Wiki corpus.
Future work may involve refining the attention mechanism or incorporating more advanced pre-trained language models like BERT to improve the detection of remote dependencies and enhance overall robustness.
Summary updated