Abstract
Information Extraction (IE) tasks have traditionally been dominated by sequential models that struggle to capture non-local dependencies. This paper introduces GraphIE, a novel framework that models data as a graph to propagate information across long-range relationships. By utilizing Graph Convolutional Networks (GCNs) within an Encoder-Graph-Decoder architecture, GraphIE significantly improves tagging accuracy and generalization across textual, social media, and visual domains.
Introduction
Most modern Information Extraction (IE) systems, such as those used for Named Entity Recognition (NER), treat text as a simple sequence of tokens. While sequence models like LSTMs are effective at capturing local context, they often fail to account for non-local dependencies. For instance, a human reader can disambiguate a mention of "Washington" later in a text by referring back to an earlier mention where it was clearly identified as a person. A standard sequential tagger, however, might miss this connection.
Beyond simple text, many IE tasks involve complex structures. In social media, a user’s attributes may be inferred from their social links. In visual documents like invoices or medical forms, the physical layout—what is positioned above or adjacent to a text box—is often more informative than the linear reading order. GraphIE addresses these challenges by allowing models to look across a graph of relationships rather than just a line of words.
Modeling Relationships as Graphs
The core innovation of GraphIE is the transformation of data into a graph structure where nodes represent units of information (words or sentences) and edges represent various types of dependencies.
As shown in the figure above, the framework can capture multiple relationship types:
- Local: Connections between adjacent tokens.
- Identical: Links between the same word appearing in different parts of a document.
- Co-referent: Links between different words referring to the same entity (e.g., "Washington" and "the president").
The GraphIE Framework
The authors propose an Encoder-Graph-Decoder architecture designed to enrich local representations with global context:
- Encoder: A standard BiLSTM (often augmented with character-level CNNs) processes the input to create context-aware hidden representations for each node.
- Graph Module: This module treats the encoder's hidden states as nodes in a graph. It uses Graph Convolutional Networks (GCNs) to iteratively propagate information between connected neighbors, allowing each node to learn from its broader context.
- Decoder: The enriched representations are passed to a BiLSTM+CRF (Conditional Random Field) decoder to produce final tags.
The framework is highly flexible, supporting both sentence-level graphs (where nodes represent entire sentences) and word-level graphs (where nodes represent individual tokens).
Experimental Evaluation
GraphIE was evaluated across three distinct domains to test its versatility:
- Textual IE: Using the CoNLL-2003 and CHEMDNER datasets, GraphIE achieved a 91.74 F1 score on CoNLL-2003, matching or exceeding state-of-the-art complex models.
- Social Media IE: On Twitter datasets (EDUCATION and JOB), the model used "followed-by" relationships to improve F1 scores by up to 3.7% over sequential baselines.
- Visual IE: On Adverse Event Case Reports (AECR), which feature complex layouts, GraphIE demonstrated massive gains in generalization. On unseen form templates, GraphIE achieved 33.7% F1, compared to just 13.4% for the sequential baseline.
Key Insights and Conclusion
The study demonstrates that graph structure acts as a powerful inductive bias. When carefully constructed graphs were replaced with random connections, performance dropped significantly, proving that the model learns from the right connections rather than just increased parameter count. GraphIE's primary strength lies in resolving ambiguity by enforcing global consistency and providing robustness to varying data layouts.
While the current framework requires manual graph construction based on domain knowledge, it paves the way for future research into automatically learning graph structures directly from data. GraphIE represents a significant step toward IE systems that can understand the complex, non-linear world of modern information.
Summary updated