A Neural Edge-Editing Approach for Document-Level Relation Graph Extraction

Kohei Makino, Makoto Miwa, Yutaka SasakiACL 2021

TL;DR

This paper introduces a novel "edge-editing" approach for extracting relation graphs from documents, treating relation extraction as an iterative refinement process. By leveraging both document content and the evolving graph structure, their method significantly improves relation extraction accuracy, especially for complex, document-spanning relationships.

What Problem Does This Paper Address?

Relation Extraction (RE) is a fundamental task in Natural Language Processing (NLP) that aims to identify relationships between entities in text. While traditional RE focused on sentence-level extraction, many real-world relations are expressed across multiple sentences within a document. Document-level RE aims to capture these broader relationships, often by constructing a graph where entities are nodes and relations are edges. However, existing document-level RE methods often struggle to effectively model the interdependencies between relations and leverage the global document context. This paper addresses the challenge of building accurate and comprehensive relation graphs from documents, particularly when relations are spread out and interconnected.

Methods

The core of this paper is a novel neural edge-editing approach. Instead of directly predicting all edges at once, their method iteratively refines an initial relation graph. This initial graph can be empty or derived from an existing system (like a rule-based extractor). The process involves an edge classifier that, for each potential edge between two entities, considers three sources of information:

  • Document Context: Encoded using a pretrained transformer model (Longformer), capturing the semantic meaning of the text.
  • Graph Context: Incorporated using a Graph Convolutional Network (GCN), which aggregates information from neighboring nodes and edges in the current state of the relation graph.
  • Current Edge Information: The existing representation of the edge being considered.

The editing process is performed in a "close-first" manner, meaning edges between entities that appear closer together in the document are processed and refined before those that are farther apart. This strategy is crucial for capturing local dependencies that can inform the extraction of more distant relations. The iterative refinement allows the model to leverage information from already edited edges, building a more coherent and accurate graph over time.

Datasets

The authors evaluate their approach on the materials science procedural text corpus (Mysore et al., 2019). This corpus is specifically annotated with a document-level graph structure, including 19 node types (e.g., materials, operations) and 15 directed relation types. The corpus consists of 200 documents for training, 15 for development, and 15 for testing.

Key Insights and Ideas

  • Iterative Edge Editing is Powerful: Treating relation extraction as an iterative refinement process, rather than a single-shot prediction, allows the model to progressively build a more accurate graph by incorporating feedback from previous edits.
  • "Close-First" Strategy Matters: Prioritizing the editing of edges between nearby entities helps capture local dependencies that are essential for understanding the overall document structure and extracting more complex, distant relations.
  • Synergy Between Rule-Based and Neural Methods: The approach effectively leverages the strengths of both rule-based systems (for initial graph generation) and neural methods (for refinement and capturing nuances), leading to superior performance.
  • Graph Structure as a Rich Context: Incorporating graph convolutional networks to capture the context of nodes within the evolving graph significantly enhances the edge classifier's ability to make accurate predictions.
  • Document-Level Context is Crucial: Using Longformer for document encoding highlights the importance of considering the entire document's context for disambiguating and identifying relations that span sentences.

Key Results

The proposed EDIT (Edge-Editing) approach achieved a micro F-score of 86.3% for edge prediction on the materials science procedural text corpus.

  • When initialized with empty graphs, EDIT outperformed baselines like EDIT-IE (no iterative editing) and RANDOM EDIT (random editing order), demonstrating the effectiveness of both iterative editing and the close-first strategy.
  • When initialized with a rule-based model (RULE), EDIT achieved an F-score of 0.851, which was competitive with and sometimes surpassed the rule-based model itself (0.807). This indicates that the neural editing process can refine and improve upon rule-based extractions.
  • Interestingly, EDIT-IE (without iterative editing but with RULE) achieved the highest score on the test set (0.863), suggesting that for this specific dataset and initialization, a single pass of editing might be sufficient or even optimal. However, the overall EDIT approach still showed strong performance and the ability to capture relations missed by RULE.

Limitations and Future Work

  • Dataset Specificity: The evaluation was primarily conducted on a materials science procedural text corpus. While this corpus is rich in document-level relations, the generalizability of the approach to other domains with different relation structures and document complexities remains to be fully explored.
  • Computational Cost: While the paper mentions reducing computational cost by grouping pairs with the same distance, iterative editing can still be computationally intensive.
  • Future Work: The authors plan to improve the approach for more consistent and accurate relation graphs. They also express interest in applying their method to other datasets, such as cooking recipes and temporal graphs, to further validate its versatility.

Summary updated

BibTeX
@inproceedings{makino2021neural,
  author = {Kohei Makino and Makoto Miwa and Yutaka Sasaki},
  title = {{A Neural Edge-Editing Approach for Document-Level Relation Graph Extraction}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
}
Figure