A discovery and verification approach to pharmacovigilance using electronic healthcare data

Louis Dijkstra, Tania Schink, Roland Linder, Markus Schwaninger, Iris Pigeot, Marvin N. Wright, Ronja ForaitaFrontiers in Pharmacology 2024 · DOI

TL;DR

This paper introduces a novel, ensemble-based approach to pharmacovigilance that leverages electronic healthcare data (EHC) for more robust adverse drug reaction (ADR) signal detection. By combining multiple signal detection methods using a Borda count and offering a verification phase, this method aims to improve the accuracy and efficiency of identifying potential drug safety issues, with an accompanying online tool for practical application.

What Problem Does This Paper Address?

Ensuring drug safety after a medication is approved and in widespread use is crucial. Traditional pharmacovigilance relies on spontaneous reporting systems (SRSs), which can suffer from under-reporting and bias. While electronic healthcare (EHC) data offers a richer source of information (drug prescriptions, medical events, patient demographics), effectively extracting safety signals from this complex data remains a challenge. Existing methods often use a single signal detection algorithm, which might miss signals or generate false positives due to the inherent limitations of each method. Furthermore, verifying potential signals is a time-consuming process. This paper addresses the need for a more comprehensive and reliable system for discovering and verifying potential adverse drug reactions (ADRs) from EHC data.

Methods

The proposed approach enhances the conventional pharmacovigilance process in three key ways:

  1. Ensemble Signal Detection: Instead of relying on a single signal detection method, the authors propose using an ensemble of multiple methods. This leverages the strengths of different algorithms and mitigates their individual weaknesses. The paper specifically mentions using Bayesian confidence propagation neural network (BCPNN), longitudinal Gamma Poisson shrinker (LGPS), penalized regression (LASSO), and random forests (RFs).
  2. Borda Count Ranking: To combine the results from the multiple signal detection methods, a Borda count is employed. This method, borrowed from electoral mathematics, aggregates the rankings from each individual method into a single, unified ranking. The Borda count emphasizes consensus, meaning a drug-ADR pair that is consistently ranked highly by multiple methods will receive a better overall score.
  3. Verification Phase: After signal detection and triage by a committee of experts, the approach includes an optional verification phase. This involves conducting tailored pharmacoepidemiological studies (e.g., nested case-control studies) to confirm or refute the plausibility of identified signals, especially for less common or novel ADRs.

The authors also developed an online tool (https://borda.bips.eu) to facilitate the application of their Borda count ranking method.

Datasets

The primary dataset used in this study is the German Pharmacoepidemiological Research Database (GePaRD). This database contains claims data from four statutory health insurance providers in Germany, covering approximately 25 million individuals insured since 2004. For the case study, a specific cohort was selected, including individuals insured with Techniker Krankenkasse (TK) for at least 12 months between January 2015 and December 2016, who had at least one dispensation of rivaroxaban (RVX) and no prior DOAC use. The verification study utilized data from 2011 to 2017.

Key Insights and Ideas

  • Ensemble Power: Combining multiple signal detection methods is more robust than using a single one. Different methods have different strengths (e.g., BCPNN for innocent bystanders, LASSO for handling multiple ADRs and confounders, RFs for non-linear relationships), and their combined output provides a more balanced view.
  • Consensus is Key: The Borda count effectively captures consensus among signal detection methods. This is crucial because a signal that appears strong in one method but weak in others might be a false positive, whereas a consistently ranked signal across methods is more likely to be genuine.
  • Verification is Essential but Costly: While EHC data and ensemble methods can efficiently generate many potential signals, expert review and targeted verification studies are vital to confirm true safety concerns and avoid hasty conclusions. However, verification is time-consuming and resource-intensive, necessitating careful selection of signals for this phase.
  • EHC Data's Dual Role: EHC data is not only valuable for signal detection but also for signal verification, enabling a more integrated pharmacovigilance workflow.

Key Results

The case study focused on rivaroxaban (RVX), a direct oral anticoagulant.

  • Signal Detection: The ensemble method successfully identified known adverse drug reactions (ADRs) associated with RVX, such as gastrointestinal hemorrhage (K92.2), which was ranked highly. It also flagged potential new signals.
  • Signal Triage: A committee of experts identified four health outcomes of interest for further investigation: acute liver injury (ALI), acute cystitis (CYS), epilepsy and seizures (EPI), and sepsis. Gastrointestinal bleeding (GB) and intracranial bleeding (ICB) were used as positive controls.
  • Verification Study:
    • Positive Controls: The study confirmed increased risks for gastrointestinal bleeding (OR = 1.37) and a decreased risk for intracranial bleeding (OR = 0.74) with RVX compared to phenprocoumon (PPC), aligning with existing literature.
  • New Signals: A slightly increased risk for acute cystitis (OR = 1.11) was observed.
  • No Significant Safety Issues: For acute liver injury (OR = 0.50) and sepsis (OR = 1.10), the results did not indicate significant safety concerns when compared to PPC.
  • Confirmed Elevated Risk: The verification study confirmed an elevated risk for epilepsy and seizures in RVX users compared to PPC users (OR = 1.26), a finding that was further investigated and published separately.

Limitations and Future Work

  • Verification Time and Cost: The authors acknowledge that conducting pharmacoepidemiological verification studies is laborious and time-consuming, limiting the number of signals that can be practically verified.
  • Data Splitting: While it's standard practice to split EHC data into detection and verification sets to avoid overfitting, this can reduce the power to detect rare signals or ADRs, especially with limited data. The authors suggest this trade-off needs careful consideration.
  • Data Lag: EHC data can have a significant delay between event occurrence and availability, making it challenging to detect ADRs for very new drugs. SRS data might be more suitable for such early-stage detection.
  • Single Drug Focus: The case study focused on a single medication (rivaroxaban). While the approach is scalable to multiple drugs, the resulting Borda ranking would be longer.
  • Future Work: The authors suggest further research is needed to explore how to extend their automated triage process to encompass more drugs and to investigate the optimal set of signal detection methods for an ensemble. They also highlight the need for careful consideration of data splitting strategies when data is limited. The development of the online tool is a step towards facilitating wider adoption and application of their approach.

Summary updated

Abstract

The case study reveals our ability to detect known adverse drug reactions and discover new signals. Importantly, the ensemble method is computationally efficient. Hasty false conclusions can be avoided by a verification study, which is, however, time-consuming to carry out. We provide an online tool for easy application: https://borda.bips.eu.
BibTeX
@inproceedings{dijkstra2024discovery,
  author = {Louis Dijkstra and Tania Schink and Roland Linder and Markus Schwaninger and Iris Pigeot and Marvin N. Wright and Ronja Foraita},
  title = {{A discovery and verification approach to pharmacovigilance using electronic healthcare data}},
  booktitle = {Frontiers in Pharmacology},
  year = {2024},
  doi = {10.3389/fphar.2024.1426323},
}
Figure