Introduction
Semantic dependency parsing aims to map natural language sentences into graph-based representations of meaning. However, the field is fragmented across various frameworks such as DM (DELPH-IN MRS), PSD (Prague), and UCCA (Universal Conceptual Cognitive Annotation), each utilizing distinct graph structures and labeling conventions.
The Cross-Framework Meaning Representation Parsing (MRP 2019) shared task challenged researchers to develop unified models capable of parsing across these diverse frameworks. Researchers from Jeonbuk National University (JBNU) addressed this by proposing a system based on multi-task learning (MTL) with a shared encoder and framework-specific decoders, featuring a novel "multi-level" biaffine attention mechanism.
System Architecture
The JBNU system architecture is divided into a shared encoder and framework-specific decoders.
The Shared Encoder
The encoder utilizes a BERT-BiLSTM stack to handle the input text. Because BERT operates on wordpieces, the system first aggregates these into word-level embeddings using a BiLSTM. This is followed by a second, 3-layer BiLSTM that generates the final sentence representations used by all downstream tasks.
Multi-level Biaffine Decoder
The core contribution of the JBNU team is the exploration of multi-level biaffine attention. While standard biaffine decoders (based on Dozat and Manning) typically use the final layer of an encoder, the JBNU team implemented a fusion function to combine representations from all intermediate BiLSTM layers.
The rationale behind this approach is that "shallow" semantic arcs might be better captured by lower network layers, while "deep" or more abstract arcs require the complex representations of higher layers. By fusing these levels, the model theoretically gains access to a richer feature set for resolving complex dependencies.
Key Insights and Results
The team evaluated several model variants, including a baseline Biaffine model, a BERT-enhanced version, and the Multi-level MTL version.
- The Impact of BERT: Integrating BERT provided the most significant performance boost. For the UCCA framework, the Unlabeled F1 score improved by approximately 5% compared to the baseline.
- Multi-task Learning (MTL) Challenges: Surprisingly, sharing the encoder across frameworks only yielded minor improvements for UCCA and did not significantly benefit DM or PSD. This suggests that naive encoder sharing may be insufficient due to conflicting gradients between frameworks with different head/dependent definitions.
- Competition Performance: In the official MRP 2019 rankings, the JBNU system proved highly competitive, achieving 2nd place for Labeled F1 on the PSD framework and 5th place on the DM framework.
Conclusion and Future Work
While the BERT+Biaffine architecture proved robust, the multi-level fusion and MTL strategies showed that simple architectural sharing has limits. The authors noted that the fusion method itself requires further refinement to consistently outperform single-layer models. Future research will focus on more sophisticated fusion functions and task-specific "private" encoders to better manage the unique structural requirements of different semantic frameworks.
Summary updated