Paths to Relation Extraction through Semantic Structure

Jonathan Yellin, Omri AbendACL 2021 · DOI

Introduction

Recent advancements in neural Relation Extraction (RE) have increasingly relied on integrating explicit linguistic structures to supplement end-to-end models like BERT. This paper demonstrates that incorporating semantic structures—specifically Universal Conceptual Cognitive Annotation (UCCA)—significantly outperforms models relying solely on syntactic structures. By introducing a novel "path-to-root" embedding technique to represent complex Directed Acyclic Graphs (DAGs), the authors achieve a notable 1.05 F1 point improvement over standard graph-based baselines on the TACRED dataset.

The Shift from Syntax to Semantics

Relation Extraction involves identifying semantic links between entities, such as "founded_by" or "city_of_death." While many hybrid models utilize syntactic structures like Universal Dependencies (UD) via Graph Convolutional Networks (GCNs), syntax primarily describes the formal arrangement of a sentence.

In contrast, semantic frameworks like UCCA capture the underlying meaning and roles of words. However, UCCA structures are represented as complex DAGs with non-terminal nodes, making them significantly more difficult to integrate into standard neural architectures than simple syntactic trees.

Comparison of UD (top) and UCCA (bottom) parse graphs for the same sentence.

Methodology: Integrating Semantic DAGs

The authors build upon the Contextualized Graph Convolution Network (C-GCN) framework, proposing two primary methods for semantic integration:

1. Bilexical Reduction

To adapt UCCA for standard GCNs, the authors simplify the complex graph into a "bilexical" format. In this version, edges exist only between terminal nodes (actual words), allowing the semantic graph to function as an adjacency matrix within the network.

2. Path-to-Root Embeddings

The paper's most significant contribution is the "path-to-root" embedding. To preserve the rich information in UCCA's non-terminal nodes without flattening the graph, every word is mapped to its unique path leading to the root of the semantic DAG. These paths are encoded into 80-dimensional vectors and concatenated directly onto the word embeddings. This allows the model to perceive the full semantic context of a word as a feature of the token itself.

Visualization of the path-to-root embedding technique, mapping token paths to vector representations.

Experimental Results

The models were evaluated on the TACRED dataset, which contains over 106,000 sentences across 42 relation types. Since TACRED lacks native semantic annotations, the authors re-annotated the dataset using the TUPA UCCA parser.

Key performance metrics include:

  • Baseline (UD only): 66.27 F1
  • UCCA Bilexical only: 66.44 F1
  • Combined (UD + UCCA + Consecutive): 66.94 F1
  • Full Model (Combined + Path Embeddings): 67.32 F1
  • Ensemble Model: 68.41 F1

The improvement from 66.27 to 67.32 F1 is statistically significant, suggesting that semantic paths provide a clearer "map" for the model than syntactic ones alone.

Insights and Analysis

The study reveals that syntactic and semantic structures possess complementary strengths. UCCA excels at identifying location-based relations (e.g., country_of_birth) because it is highly sensitive to the "events" binding entities. Conversely, UD remains superior for family relations (e.g., per:children), which often follow more rigid syntactic patterns.

Furthermore, the path-to-root method demonstrates that complex graph structures can be utilized without lossy simplification. By treating the graph structure as a token feature, the authors bypass the architectural limitations inherent in standard GCNs.

Conclusion and Future Work

Integrating semantic structures via path-to-root embeddings offers a robust path forward for Relation Extraction. While the computational overhead of UCCA parsing remains a challenge for real-time applications, the performance gains are undeniable. Future research may explore extending this path-embedding method to other semantic representations, such as Abstract Meaning Representation (AMR), to determine if different semantic "flavors" yield further improvements.

Summary updated

BibTeX
@inproceedings{yellin2021paths,
  author = {Jonathan Yellin and Omri Abend},
  title = {{Paths to Relation Extraction through Semantic Structure}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.18653/v1/2021.findings-acl.231},
}
Figure