GDPNet: Refining Latent Multi-View Graph for Relation Extraction

Fuzhao Xue, Aixin Sun, Hao Zhang, Eng Siong Chng

Introduction

Relation Extraction (RE) is the task of identifying semantic relationships between entity pairs within a text—for instance, determining that "Frank" is the "brother" of "S2" based on a conversation. A significant hurdle in RE, particularly in multi-turn dialogues, is the presence of irrelevant "noise" that obscures meaningful connections. While Transformer-based models like BERT provide strong contextual embeddings, they often struggle to aggregate information across long sequences or rely on error-prone external linguistic parsers.

GDPNet (Gaussian Dynamic Time Warping Pooling Net) addresses these issues by constructing a latent multi-view graph and refining it through a specialized pooling mechanism to isolate "trigger words" essential for relation prediction.

Methodology

The GDPNet architecture consists of three primary stages designed to transform raw text into a refined relational representation:

  1. BERT Module: The model utilizes BERT (or a speaker-aware variant, BERTs) to generate initial vector representations for every token in the sequence.
  2. Gaussian Graph Generator (GGG): Instead of simple similarity scores, GGG represents each token as a set of Gaussian distributions. It calculates edge weights using asymmetric Kullback-Leibler (KL) divergence, creating a directed multi-view graph that captures complex, non-reciprocal relationships between words.
  3. Dynamic Time Warping Pooling (DTWPool): This core innovation iteratively prunes the noisy graph. It employs an adaptive pooling ratio to select the most informative nodes and a SoftDTW loss function to ensure that the structural and sequential integrity of the original text is preserved even as the representation is compressed.
The GDPNet architecture, illustrating the flow from BERT embeddings through the Gaussian Graph Generator and iterative DTWPool layers to the final classifier.

Experimental Results

The model was evaluated on two major benchmarks: DialogRE, a human-annotated dialogue-level dataset, and TACRED/TACRED-Revisit, a large-scale sentence-level dataset.

GDPNet achieved state-of-the-art results on DialogRE with an F1 score of 64.9%, a significant improvement over the previous best of 61.2%. On TACRED-Revisit, it reached an F1 of 80.2%, surpassing models like SpanBERT and KnowBERT without requiring external knowledge bases. Ablation studies confirmed that both the GGG and DTWPool components are critical to this performance.

Analysis of Long-Context Performance

A key finding is that GDPNet's performance advantage scales with text length. In dialogues exceeding 400 tokens, where standard models often lose accuracy due to excessive noise, GDPNet maintains high performance. Quantitative analysis shows that DTWPool is twice as likely to retain manually identified "trigger words" compared to general tokens, effectively filtering out the noise that plagues long-form dialogue.

Comparison of F1 scores between BERTs and GDPNet across different token counts, highlighting GDPNet's superior performance in long-context scenarios.

Conclusion

GDPNet demonstrates that latent graph refinement is a powerful strategy for Relation Extraction in noisy, long-form contexts. By focusing on trigger words and maintaining sequential structure through DTWPool, it overcomes the limitations of traditional sequence models. Future work may involve adapting this architecture to other document-level NLP tasks where long-range dependencies and noise reduction are critical, such as summarization or complex question answering.

Summary updated

Abstract

Relation Extraction (RE) is to predict the relation type of two entities that are mentioned in a piece of text, e.g., a sentence or a dialogue. When the given text is long, it is challenging to identify indicative words for the relation prediction. Recent advances on RE task are from BERT-based sequence modeling and graph-based modeling of relationships among the tokens in the sequence. In this paper, we propose to construct a latent multi-view graph to capture various possible relationships among tokens. We then refine this graph to select important words for relation prediction. Finally, the representation of the refined graph and the BERT-based sequence representation are concatenated for relation extraction. Specifically, in our proposed GDPNet (Gaussian Dynamic Time Warping Pooling Net), we utilize Gaussian Graph Generator (GGG) to generate edges of the multi-view graph. The graph is then refined by Dynamic Time Warping Pooling (DTWPool). On DialogRE and TACRED, we show that GDPNet achieves the best performance on dialogue-level RE, and comparable performance with the state-of-the-arts on sentence-level RE. Our code is available at https://github.com/XueFuzhao/GDPNet.
BibTeX
@inproceedings{xue2021gdpnet,
  author = {Fuzhao Xue and Aixin Sun and Hao Zhang and Eng Siong Chng},
  title = {{GDPNet: Refining Latent Multi-View Graph for Relation Extraction}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i16.17670},
}
Figure