TL;DR
This paper introduces a standardized statistical pipeline to compare the safety of multiple hepatitis C drugs using post-market data from FAERS and EHR. The study found statistically significant differences in adverse event (AE) rates between older and newer hepatitis C therapies, though the findings from the two data sources were not entirely consistent, highlighting the need for sophisticated methods to synthesize evidence from diverse data.
What Problem Does This Paper Address?
Post-market surveillance of drug safety is crucial for understanding potential harms that may not be detected in pre-market clinical trials. Hepatitis C, a significant global health issue, has seen substantial therapeutic advancements, leading to high cure rates. However, ensuring the safety of these new treatments remains a concern. While data from the FDA Adverse Event Reporting System (FAERS) and Electronic Health Records (EHR) are valuable, there's a lack of standardized statistical procedures to systematically compare drug safety across multiple medications using these disparate data sources. This makes it challenging to draw consistent conclusions and support regulatory decision-making.
Methods
The authors propose a three-step statistical procedure for comparing drug safety:
- Descriptive Comparison and Visualization: This initial step involves comparing the distribution of the total number of adverse events (AEs) per report or patient between treatment groups using histograms. This provides a general overview of safety profiles.
- Testing for Difference in AE Rates: A chi-squared test is employed to assess statistical significance in AE reporting rates across treatment groups. Bonferroni correction is used to control for multiple comparisons, ensuring a stringent significance level.
- Quantification and Visualization of Effect Sizes: To address potential biases from large sample sizes and to understand the magnitude of differences, logistic regression models are used. These models quantify the odds ratios of experiencing AEs between treatment groups, both unadjusted and adjusted for patient characteristics like age and gender. This step helps identify which differences are truly clinically meaningful and how much they can be attributed to patient factors.
Datasets
- FAERS (FDA Adverse Event Reporting System): Spontaneous AE reports submitted between January 1, 2004, and December 31, 2015. The dataset was normalized by removing duplicate records and mapping drug names to RxNorm.
- Cerner Health Facts EHR Database: Longitudinal EHR data from multiple Cerner and non-Cerner facilities, collected between 1999 and 2015. This dataset includes diagnosis codes, lab results, hospital admissions, and medication data.
Key Insights and Ideas
- Data Source Discrepancies: The study highlights that FAERS and EHR data, while both valuable, can yield different conclusions regarding drug safety. This underscores the complexity of synthesizing evidence from heterogeneous sources.
- The Importance of Standardization: The proposed pipeline offers a much-needed standardized approach to comparing drug safety across multiple drugs and data sources, facilitating more robust and reproducible research.
- Patient Characteristics Matter: Age and gender can significantly influence AE reporting rates, and accounting for these factors is crucial for a more accurate assessment of drug safety.
- FAERS vs. EHR Strengths/Weaknesses: FAERS excels in capturing a broad range of spontaneous reports, while EHR provides richer clinical context and temporal information, but both have limitations in data quality and standardization.
Key Results
- The study found statistically significant differences in AE rates between older (pre-2011) and newer (post-2013) hepatitis C therapies.
- In FAERS, patients on older drugs generally reported more AEs than those on newer drugs. Conversely, in the Cerner Health Facts EHR data, patients on newer drugs tended to have more AE encounters.
- For 12 out of 30 studied AEs, patient characteristics (age and gender) explained between 0.2% and 14.8% of the difference in odds ratios between the new and old treatment groups.
- The top 5 AEs with the largest difference between adjusted and unadjusted odds ratios varied between FAERS and Cerner Health Facts EHR, further illustrating the data source discrepancies. For instance, "Ascites" showed a notable difference in FAERS, while "Weight decreased" did in Cerner EHR.
Limitations and Future Work
- Inconsistent Findings: The study acknowledges the lack of high consistency between FAERS and Cerner Health Facts EHR data, attributing this to differences in data collection mechanisms, data elements, terminologies, and sampling biases.
- Limited Temporal Information: The accuracy of temporal information in EHR data can be affected by medical visit schedules, potentially biasing AE comparisons.
- Lack of "Never-Exposed" Control Group: Both FAERS and EHR data lack a true "never-exposed" control group, which is ideal for pharmacovigilance research.
- Rare Events: The proposed method may have low power for analyzing rare events due to data sparsity.
- Future Work: The authors suggest extending the method to analyze rare events and incorporating Poisson regression models to study AE rates and rate ratios, potentially using zero-inflated models. They are also developing an R package called "AEtools" to semi-automate the pipeline. Integrating EHR data from multiple hospitals or large consortiums is recommended for studying rare diseases or rare adverse reactions.
Summary updated