Explainability for NLP in Pharmacovigilance: A Study on Adverse Event Report Triage in Swedish

Luise Dürlich, Erik Bergman, Maria Larsson, Hercules Dalianis, Seamus Doyle, Gabriel Westman, Joakim NivreNAACL 2025

TL;DR

This paper investigates how well different machine learning models can explain their predictions for identifying serious adverse drug reactions (ADRs) in Swedish healthcare reports. They found that while models generally learn relevant features, the prioritization of these features varies significantly between models and explanation methods, and reporting style can heavily influence predictions. This highlights the need for more nuanced approaches to explainability in safety-critical domains like pharmacovigilance.

What Problem Does This Paper Address?

Pharmacovigilance (PV) is crucial for drug safety, involving the detection and assessment of adverse drug reactions (ADRs). As the volume of ADR reports grows, automation is becoming essential. However, regulatory compliance and trust in AI systems demand that these automated systems are not just accurate but also explainable. This paper tackles the challenge of understanding why AI models flag an ADR report as "serious" or "non-serious." Specifically, it explores how different AI models, trained for the same task, generate explanations, whether these explanations align with regulatory definitions of seriousness, and if there are systematic patterns in misclassified reports that reveal underlying issues with the data or models.

Methods

The researchers employed two primary post-hoc feature attribution methods: Integrated Gradients (IG) and Expected Gradients (EG). These methods aim to quantify the contribution of each word (or subword token) in a report to the model's final prediction. They applied these methods to four pre-trained transformer models (KB-BERT, SweDeClin-BERT, AER-BERT, and GPT-SW3) fine-tuned for the binary classification task of triaging Swedish ADR reports into "serious" or "non-serious." To analyze the explanations, they aggregated token-level attributions to the word level and then grouped terms into meaningful clusters (e.g., based on regulatory criteria or medical concepts) to identify patterns. They also conducted a manual analysis of misclassified reports to uncover systematic issues.

Datasets

The study utilized a dataset of Swedish Adverse Event Reports (AERs) collected by the Swedish Medical Products Agency (MPA). This dataset was annotated for seriousness by expert assessors. The authors used a training, development, and test split, noting that they worked with an improved version of the data compared to previous studies, which involved removing certain comments and handling truncated reports.

Key Insights and Ideas

  • Model Priorities Differ: Even when trained on the same data for the same task, different models (and even different explanation methods applied to the same model) prioritize different features when deciding if a report is serious. This suggests that "explainability" can be model-specific.
  • Reporting Style Matters: The way a report is written—by consumers versus healthcare professionals—significantly impacts how models interpret it. Form patterns (e.g., specific phrasing from reporting forms) and temporal references were often explained as non-serious, potentially due to stylistic differences rather than the actual medical content.
  • Regulatory Alignment is Partial: While models generally learn to associate terms related to regulatory criteria for seriousness (like "death" or "hospitalization") with serious predictions, the relative importance assigned to these criteria varies.
  • Data Quality is a Hurdle: Manual reannotation of misclassified reports revealed that over half of the labels changed, highlighting potential issues with the original "gold standard" labels. This suggests that even expert annotations can be subjective or influenced by reporting guidelines, impacting model training and evaluation.

Key Results

  • All four models (KB-BERT, SweDeClin-BERT, AER-BERT, GPT-SW3) learned to identify relevant concepts for serious ADRs.
  • Correlations between global explanations across models were generally weak to moderate, especially between different model architectures (BERT vs. GPT).
  • Clusters of terms related to extreme situations (e.g., suicide, ambulance), organ-specific issues, and common symptoms were identified as important by most models.
  • Manual analysis of misclassified reports indicated that stylistic differences in reporting (e.g., consumer vs. healthcare professional language) and the way information is presented (e.g., form patterns) can strongly influence model predictions, sometimes leading to misclassification.
  • Recall and precision were consistently lower for consumer reports compared to healthcare professional reports, even after label correction.

Limitations and Future Work

The authors acknowledge several limitations:

  • Feature Attribution Limitations: These methods highlight important features but don't fully explain why the model relies on them or how features interact.
  • Binary Classification Simplification: The binary "serious/non-serious" classification might oversimplify the nuances of ADR assessment, and future work could explore how to better define and contrast these classes.
  • Data Nuances: The influence of reporting style and potential issues with gold labels suggest that future work should investigate how to train models that are less susceptible to these confounding factors and potentially develop more robust annotation procedures.
  • Model Training: The choice of a generative model (GPT-SW3) with a classification setup might not be optimal, and reframing the task into a generative setup could be explored.

Future work could focus on developing explanation methods that better capture feature interactions, refining the definition of "serious" for AI models, and exploring more advanced training strategies to mitigate the impact of reporting style and improve robustness.

Summary updated

BibTeX
@inproceedings{durlich2025explainability,
  author = {Luise Dürlich and Erik Bergman and Maria Larsson and Hercules Dalianis and Seamus Doyle and Gabriel Westman and Joakim Nivre},
  title = {{Explainability for NLP in Pharmacovigilance: A Study on Adverse Event Report Triage in Swedish}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure