Introduction
Relation Extraction (RE) is the task of identifying semantic relationships between entity pairs within a text—for instance, determining that "Frank" is the "brother" of "S2" based on a conversation. A significant hurdle in RE, particularly in multi-turn dialogues, is the presence of irrelevant "noise" that obscures meaningful connections. While Transformer-based models like BERT provide strong contextual embeddings, they often struggle to aggregate information across long sequences or rely on error-prone external linguistic parsers.
GDPNet (Gaussian Dynamic Time Warping Pooling Net) addresses these issues by constructing a latent multi-view graph and refining it through a specialized pooling mechanism to isolate "trigger words" essential for relation prediction.
Methodology
The GDPNet architecture consists of three primary stages designed to transform raw text into a refined relational representation:
- BERT Module: The model utilizes BERT (or a speaker-aware variant, BERTs) to generate initial vector representations for every token in the sequence.
- Gaussian Graph Generator (GGG): Instead of simple similarity scores, GGG represents each token as a set of Gaussian distributions. It calculates edge weights using asymmetric Kullback-Leibler (KL) divergence, creating a directed multi-view graph that captures complex, non-reciprocal relationships between words.
- Dynamic Time Warping Pooling (DTWPool): This core innovation iteratively prunes the noisy graph. It employs an adaptive pooling ratio to select the most informative nodes and a SoftDTW loss function to ensure that the structural and sequential integrity of the original text is preserved even as the representation is compressed.
Experimental Results
The model was evaluated on two major benchmarks: DialogRE, a human-annotated dialogue-level dataset, and TACRED/TACRED-Revisit, a large-scale sentence-level dataset.
GDPNet achieved state-of-the-art results on DialogRE with an F1 score of 64.9%, a significant improvement over the previous best of 61.2%. On TACRED-Revisit, it reached an F1 of 80.2%, surpassing models like SpanBERT and KnowBERT without requiring external knowledge bases. Ablation studies confirmed that both the GGG and DTWPool components are critical to this performance.
Analysis of Long-Context Performance
A key finding is that GDPNet's performance advantage scales with text length. In dialogues exceeding 400 tokens, where standard models often lose accuracy due to excessive noise, GDPNet maintains high performance. Quantitative analysis shows that DTWPool is twice as likely to retain manually identified "trigger words" compared to general tokens, effectively filtering out the noise that plagues long-form dialogue.
Conclusion
GDPNet demonstrates that latent graph refinement is a powerful strategy for Relation Extraction in noisy, long-form contexts. By focusing on trigger words and maintaining sequential structure through DTWPool, it overcomes the limitations of traditional sequence models. Future work may involve adapting this architecture to other document-level NLP tasks where long-range dependencies and noise reduction are critical, such as summarization or complex question answering.
Summary updated