Abstract
Document-level models for information extraction tasks like slot-filling are flexible: they can be applied to settings where information is not necessarily localized in a single sentence. For example, key features of a diagnosis in a radiology report may not be explicitly stated in one place, but nevertheless can be inferred from parts of the report's text. However, these models can easily learn spurious correlations between labels and irrelevant information. This work studies how to ensure that these models make correct inferences from complex text and make those inferences in an auditable way: beyond just being right, are these models "right for the right reasons?" We experiment with post-hoc evidence extraction in a predict-select-verify framework using feature attribution techniques. We show that regularization with small amounts of evidence supervision during training can substantially improve the quality of extracted evidence. We evaluate on two domains: a small-scale labeled dataset of brain MRI reports and a large-scale modified version of DocRED (Yao et al., 2019) and show that models' plausibility can be improved with no loss in accuracy.
Introduction
Document-level information extraction (IE) tasks, such as slot-filling, require models to synthesize information across multiple sentences. This is particularly critical in domains like radiology, where a diagnosis might be inferred from a collection of subtle clinical findings rather than a single explicit statement. While Transformer-based models achieve high accuracy in these tasks, they often function as "black boxes," potentially relying on spurious correlations or dataset biases.
In high-stakes applications, being "right" is not enough; a model must be "right for the right reasons." This paper investigates how to ensure document-level models make inferences that are both faithful (the extracted evidence actually drives the prediction) and plausible (the evidence aligns with human expert reasoning).
Methodology: Predict-Select-Verify
The authors propose a framework using a RoBERTa-based classifier to extract and verify evidence post-hoc.
Interpretation and Extraction
To identify which parts of a document contribute most to a prediction, the researchers utilize DeepLIFT, a feature attribution method that ranks sentences by their importance. To ensure faithfulness, they introduce the SUFFICIENT extraction method. This technique iteratively feeds the model the highest-ranked sentences until the prediction matches the output generated from the full document. This identifies the minimal subset of text sufficient for the model's decision.
Regularization with Evidence Supervision
To improve plausibility, the models are trained with small amounts of labeled evidence (e.g., 10% of the dataset) using two novel regularization techniques:
- Attention Regularization (ATTN): This objective encourages the model's internal attention mechanisms to focus on human-labeled evidence sentences during training.
- Entropy Maximization (ENTROPY): This technique trains the model to be uncertain (maximizing prediction entropy) when known evidence is removed. This prevents the model from "cheating" by finding alternative, non-causal cues in the remaining text.
Experimental Setup
The framework was evaluated on two datasets:
- Brain MRI Reports: A specialized dataset of radiology reports with document-level labels (e.g., "mass effect") and expert-annotated evidence.
- Adapted DocRED: A large-scale relation extraction dataset modified for document-level classification to test scalability.
Key Results and Insights
The experiments demonstrate that incorporating evidence supervision significantly improves model interpretability without compromising performance.
Plausibility and Accuracy
A major finding is that forcing models to be plausible does not incur an accuracy penalty. In the Brain MRI dataset, the SUFFICIENT_both model (combining ATTN and ENTROPY) improved evidence precision for "mass effect" from 16.5% (standard model) to 77.8%. Crucially, this was accompanied by an increase in the label F1 score from 42.1 to 60.0, suggesting that learning the "right reasons" can lead to better generalization.
Implicit Reasoning
The medical domain often involves implicit evidence. For instance, "mass effect" might be inferred from "downward displacement of the brain stem." Regularized models proved much more capable of capturing these implicit relationships. Furthermore, the study showed that even "pseudo-labels" generated by simple rule-based systems could effectively guide complex Transformers toward more human-like reasoning when expert labels are scarce.
Conclusion
This work provides a path toward more auditable and reliable document-level IE. By combining feature attribution with targeted regularization, models can be trained to provide justifications that are both faithful to their internal logic and plausible to human experts. While these rationales are approximations and should assist rather than replace human judgment, they represent a significant step toward transparent AI in critical fields like medicine.
Summary updated