Abstract
Dialogue-based Relation Extraction (RE) is a complex task that requires models to navigate conversational dynamics, track speaker identities, and pinpoint specific evidence across multiple turns. We present a multi-task learning framework that enhances a BERT-based encoder with two auxiliary tasks: Speaker Prediction and Trigger Words Prediction. These tasks encourage the model to capture speaker-specific linguistic patterns and identify the "trigger words" that signal relationships. Our approach achieves a state-of-the-art F1 score of 65.5% on the DialogRE dataset, significantly outperforming existing baselines.
Introduction
Relation Extraction (RE) involves identifying the semantic relationship between two entities in a text. While traditional RE focuses on structured documents or single sentences, dialogue-based RE presents unique challenges. Relations are often distributed across several conversational turns, and the entities involved are frequently the speakers themselves.
Existing models often treat dialogue as flat text, failing to account for the "persona" of a speaker—their unique tone or habits—or the specific "trigger words" (e.g., "Mom," "boss") that provide definitive evidence for a relationship. This paper addresses these gaps by forcing the model to explicitly learn these conversational features through auxiliary supervision.
Methodology
The proposed framework utilizes a shared BERT encoder to perform three simultaneous tasks:
- Relation Extraction (Main Task): The model predicts the relationship between two arguments using an Integration Module. This module, a two-layer highway network, concatenates global context from the
[CLS]token with local features derived from max-pooled representations of the target arguments. - Speaker Prediction (Auxiliary): To learn distinctive linguistic patterns, the model randomly masks speaker names (e.g., "S1") and attempts to predict the identity of the speaker for a given utterance.
- Trigger Words Prediction (Auxiliary): Using BIO tagging, the model identifies words within the dialogue that serve as evidence for the relation. During training, the predicted relation type is used as an additional input to guide the search for evidence.
By training on these tasks concurrently, the BERT encoder develops a more robust understanding of how conversational structure and word choice signal social or professional connections.
Experimental Setup
The model was evaluated on DialogRE, the first human-annotated dataset for dialogue-based RE. Derived from transcripts of the TV show Friends, the dataset contains 1,788 dialogues and 36 relation types. Notably, 49.6% of the relation triples in DialogRE are annotated with trigger words, providing the necessary supervision for the auxiliary trigger prediction task.
Results and Discussion
The proposed model demonstrates superior performance compared to strong baselines, including speaker-aware BERT (BERTs) and graph-based models like GDPNet.
Key Findings
- State-of-the-Art Performance: The model achieved an F1 score of 65.5% and an F1c score of 60.5% on the test set. The F1c metric is a stricter measure that evaluates how early a model can identify a relation within a dialogue.
- Ablation Analysis: Removing the Trigger Words Prediction task resulted in the most significant performance decline (a 2.0% drop in F1), highlighting that identifying explicit evidence is the most critical component of the extraction process.
- Speaker Personas: The Speaker Prediction task allows the model to learn that how someone speaks indicates who they are. This helps distinguish between similar relations (e.g., "colleague" vs. "boss") based on tone, even when explicit titles are absent.
- Structural Importance: Experiments replacing dialogue formatting with plain prose showed a significant drop in performance, confirming that the "Speaker-Utterance" structure is vital for context.
Conclusion
This work demonstrates that explicitly modeling speaker identities and identifying relational evidence significantly improves dialogue-based RE. While the model excels at identifying relations with clear triggers, some relationships remain difficult to extract due to their reliance on subtle, long-term conversational habits. Future research may explore more advanced discourse modeling or the integration of external social knowledge to handle these implicit cases.
Summary updated