Enhancing Dialogue-based Relation Extraction by Speaker and Trigger Words Prediction

Tianyang Zhao, Zhao Yan, Yunbo Cao, Zhoujun Li

Abstract

Dialogue-based Relation Extraction (RE) is a complex task that requires models to navigate conversational dynamics, track speaker identities, and pinpoint specific evidence across multiple turns. We present a multi-task learning framework that enhances a BERT-based encoder with two auxiliary tasks: Speaker Prediction and Trigger Words Prediction. These tasks encourage the model to capture speaker-specific linguistic patterns and identify the "trigger words" that signal relationships. Our approach achieves a state-of-the-art F1 score of 65.5% on the DialogRE dataset, significantly outperforming existing baselines.

Introduction

Relation Extraction (RE) involves identifying the semantic relationship between two entities in a text. While traditional RE focuses on structured documents or single sentences, dialogue-based RE presents unique challenges. Relations are often distributed across several conversational turns, and the entities involved are frequently the speakers themselves.

Existing models often treat dialogue as flat text, failing to account for the "persona" of a speaker—their unique tone or habits—or the specific "trigger words" (e.g., "Mom," "boss") that provide definitive evidence for a relationship. This paper addresses these gaps by forcing the model to explicitly learn these conversational features through auxiliary supervision.

Methodology

The proposed framework utilizes a shared BERT encoder to perform three simultaneous tasks:

  1. Relation Extraction (Main Task): The model predicts the relationship between two arguments using an Integration Module. This module, a two-layer highway network, concatenates global context from the [CLS] token with local features derived from max-pooled representations of the target arguments.
  2. Speaker Prediction (Auxiliary): To learn distinctive linguistic patterns, the model randomly masks speaker names (e.g., "S1") and attempts to predict the identity of the speaker for a given utterance.
  3. Trigger Words Prediction (Auxiliary): Using BIO tagging, the model identifies words within the dialogue that serve as evidence for the relation. During training, the predicted relation type is used as an additional input to guide the search for evidence.
The proposed multi-task learning architecture featuring the shared BERT encoder and auxiliary prediction modules.

By training on these tasks concurrently, the BERT encoder develops a more robust understanding of how conversational structure and word choice signal social or professional connections.

Experimental Setup

The model was evaluated on DialogRE, the first human-annotated dataset for dialogue-based RE. Derived from transcripts of the TV show Friends, the dataset contains 1,788 dialogues and 36 relation types. Notably, 49.6% of the relation triples in DialogRE are annotated with trigger words, providing the necessary supervision for the auxiliary trigger prediction task.

Results and Discussion

The proposed model demonstrates superior performance compared to strong baselines, including speaker-aware BERT (BERTs) and graph-based models like GDPNet.

Performance comparison on the DialogRE dataset showing the proposed model achieving state-of-the-art results.

Key Findings

  • State-of-the-Art Performance: The model achieved an F1 score of 65.5% and an F1c score of 60.5% on the test set. The F1c metric is a stricter measure that evaluates how early a model can identify a relation within a dialogue.
  • Ablation Analysis: Removing the Trigger Words Prediction task resulted in the most significant performance decline (a 2.0% drop in F1), highlighting that identifying explicit evidence is the most critical component of the extraction process.
  • Speaker Personas: The Speaker Prediction task allows the model to learn that how someone speaks indicates who they are. This helps distinguish between similar relations (e.g., "colleague" vs. "boss") based on tone, even when explicit titles are absent.
  • Structural Importance: Experiments replacing dialogue formatting with plain prose showed a significant drop in performance, confirming that the "Speaker-Utterance" structure is vital for context.

Conclusion

This work demonstrates that explicitly modeling speaker identities and identifying relational evidence significantly improves dialogue-based RE. While the model excels at identifying relations with clear triggers, some relationships remain difficult to extract due to their reliance on subtle, long-term conversational habits. Future research may explore more advanced discourse modeling or the integration of external social knowledge to handle these implicit cases.

Summary updated

BibTeX
@inproceedings{zhao2021enhancing,
  author = {Tianyang Zhao and Zhao Yan and Yunbo Cao and Zhoujun Li},
  title = {{Enhancing Dialogue-based Relation Extraction by Speaker and Trigger Words Prediction}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.18653/v1/2021.findings-acl.402},
}
Figure