Abstract
Large Language Models (LLMs) often exhibit relation bias in Relation Extraction (RE) tasks, relying on pre-trained stereotypes rather than specific textual context. This paper introduces Supervised Rationale Verification and Feedback (SRVF), a novel framework that employs a rationale supervisor to detect biased explanations. By providing re-selected few-shot demonstrations as feedback, SRVF guides LLMs to refine their predictions. Evaluated across five major benchmarks, SRVF significantly outperforms standard In-Context Learning (ICL) and existing feedback methods, demonstrating robustness across various model scales.
Introduction
Large Language Models (LLMs) are powerful tools for natural language processing, yet they are susceptible to "relation bias"—a tendency to favor common pre-trained patterns over specific contextual evidence. For instance, an LLM might misclassify the sentence "data is derived from a study" as a "Product-Producer" relationship simply because "data" is frequently associated with human production in its training data. To address this, researchers from Wuhan University presented the Supervised Rationale Verification and Feedback (SRVF) framework at AAAI-2025.
Problem Statement: The Feedback Gap in RE
Relation Extraction (RE) involves identifying semantic relationships between entities. While LLMs excel at many tasks, they struggle with RE due to two primary factors:
- Relation Bias: Over-reliance on pre-training patterns rather than the current instance's context.
- Feedback Inadequacy: General feedback methods like Self-Refine often fail in RE because errors are subtle and semantic. Simply informing an LLM it is "wrong" is insufficient; it requires contextual demonstrations to recalibrate its reasoning.
The SRVF Framework
The SRVF framework introduces a "rationale supervisor" to identify biased logic and provide targeted feedback through re-selected demonstrations.
1. Causal Intervention for Data Collection
To train the supervisor, the authors utilized Structural Causal Models (SCM) to generate both biased and unbiased rationales. By guiding the LLM with correct labels, they induced "clean" explanations. Conversely, "diversified intervention"—intentionally providing misleading examples—was used to collect common reasoning errors for training data.
2. Training the Rationale Supervisor
A BERT-like model serves as the Supervisor, trained via contrastive learning. It learns to distinguish between biased and unbiased rationales by clustering similar biased explanations together while distancing them from correct ones in the representation space.
3. Verification and Feedback Loop
During inference, the process follows these steps:
- Initial Prediction: The LLM generates a relation label and a supporting rationale.
- Verification: The Supervisor evaluates the rationale. If flagged as biased, the process moves to feedback.
- Feedback: The system retrieves feedback demonstrations—examples semantically similar to the error but demonstrating the correct logic.
- Correction: The LLM uses these new demonstrations to produce a refined, unbiased prediction.
Experimental Evaluation
The researchers evaluated SRVF on five benchmarks: SemEval-2010 Task 8, TACRED, Re-TACRED (sentence-level), and DocRED, Re-DocRED (document-level). Testing spanned various few-shot settings (5 to 50 examples).
Key Results
- Performance Gains: SRVF improved standard In-Context Learning (ICL) by an average of 10.65% in micro-F1 score when using random initial examples.
- Superiority over Baselines: SRVF consistently outperformed other feedback frameworks like Self-Refine and GRACE. Notably, Self-Refine occasionally degraded performance compared to basic ICL in RE tasks because its feedback wasn't tailored for semantic extraction.
- Scalability: The framework proved effective across model sizes, from Llama-2-7b to Meta-Llama-3-70B-Instruct, providing a 2.47% boost even to the larger model's high baseline.
Conclusion and Future Work
SRVF demonstrates that an LLM's rationale is a critical indicator of bias. By verifying reasoning rather than just the final label, and providing demonstrations as feedback, the framework effectively mitigates relation bias. While SRVF requires labeled data for training the supervisor and adds some iterative latency, its efficiency—triggering second calls only when bias is detected—makes it a viable solution for high-accuracy RE. Future research will explore applying rationale-based feedback to other complex tasks like event extraction.
Summary updated