Natural language processing for detecting adverse drug events: A systematic review protocol

Imane Guellil, Jinge Wu, Aryo Pradipta Gema, Farah Francis, Yousra Berrachedi, Nidhaleddine Chenni, Richard Tobin, Clare Llewellyn, Stella Arakelyan, Honghan Wu, Bruce Guthrie, Beatrice Alex

TL;DR

This paper outlines a systematic review protocol designed to map the landscape of Natural Language Processing (NLP) techniques used for detecting Adverse Drug Events (ADEs) from unstructured text. By consolidating existing research, this work aims to provide a comprehensive understanding of current methods, challenges, and future directions, ultimately paving the way for improved patient safety and more effective drug monitoring.

What Problem Does This Paper Address?

Adverse Drug Events (ADEs) are a significant public health concern, leading to considerable healthcare costs and patient suffering. While ADEs are often documented in various forms of unstructured text (like clinical notes, discharge summaries, and social media posts), extracting this information efficiently and accurately is a major challenge. Traditional methods struggle with the nuances of natural language, making it difficult to identify and understand ADEs. Natural Language Processing (NLP) offers a powerful solution for automating this extraction process, but the field is rapidly evolving, with numerous approaches and tools emerging. This paper addresses the need for a structured overview of this burgeoning research area to guide future development and application of NLP for ADE detection.

Methods

The core of this paper is a detailed systematic review protocol. The authors outline a rigorous, six-stage process for identifying, screening, and analyzing relevant literature. Key methodological aspects include:

  • Comprehensive Database Search: Utilizing six major academic databases (Embase, Medline, Web of Science Core Collection, ACM Guide to Computing Literature, IEEE Digital Library, and Scopus) to ensure broad coverage.
  • Systematic Screening: Employing a multi-reviewer approach for title, abstract, and full-text screening to minimize bias, with a defined process for resolving disagreements.
  • Data Extraction: A structured data charting form, developed in consultation with experts, to systematically extract information on NLP approaches, datasets, annotation schemas, evaluation methods, and demographic details.
  • Risk of Bias Assessment: Utilizing the PROBAST tool to evaluate the quality and potential biases of the included studies.
  • Thematic Analysis and Synthesis: Planning to present findings through tabulated summaries, manual thematic analysis, and potentially meta-analysis to highlight key themes and trends.

Datasets

The paper does not introduce new datasets. Instead, it focuses on identifying and characterizing the datasets used in the studies selected for the systematic review. The data extraction process specifically aims to capture information about:

  • Data Resources: Including the name, type (corpus, lexicon, ontology), source (e.g., EHRs, social media), size, language, and accessibility of datasets.
  • Annotation Information: Details on the type of annotation (manual, automatic, semi-automatic), pre-processing steps, annotated ADEs, and annotation guidelines.

Key Insights and Ideas

  • The Growing Importance of NLP in Healthcare: The paper underscores the increasing recognition of NLP's potential to unlock valuable insights from unstructured clinical text, particularly for patient safety initiatives like ADE detection.
  • The Need for a Structured Overview: The rapid proliferation of NLP research in this domain highlights the critical need for a systematic review to consolidate knowledge, identify trends, and avoid redundant efforts.
  • Multifaceted Nature of ADE Detection: The authors anticipate that their review will reveal a diverse range of NLP tasks beyond simple extraction, including classification and normalization, and a variety of data sources from clinical notes to social media.
  • Challenges in ADE Detection: Preliminary analysis suggests common challenges such as imbalanced datasets, the ambiguity of abbreviations, and difficulties in detecting rare ADEs.

Key Results

As this paper is a protocol, the "Results" section describes the process of data collection rather than the findings of the systematic review itself.

  • Search Strategy Developed: A comprehensive search strategy was developed in consultation with domain experts and librarians.
  • Data Extraction Completed: Data extraction from 178 identified studies has been completed.
  • Writing in Progress: The authors are currently writing the main systematic review paper, synthesizing the findings from the extracted data.

Limitations and Future Work

  • Database Selection: Acknowledged limitation is the exclusion of the ACL Anthology, a significant venue for NLP research. While the authors justify their selection based on expert consultation and the inclusion of ACL-related papers found in other databases, they recognize this as a potential limitation and plan to consider ACL venues in future reviews.
  • Time Lag: Given the fast-moving nature of NLP research, the time lag between data collection (2022) and publication is a concern, which the authors are addressing by expediting the writing process.
  • Scope of Previous Reviews: The authors highlight the limitations of previous reviews (e.g., focusing on specific databases or timeframes) as a key motivation for their own comprehensive approach.
  • Future Work: The systematic review itself, once completed and published, will lay the groundwork for future research by providing a clear understanding of the current landscape, identifying effective evaluation techniques, and highlighting gaps in the literature.

Summary updated

Abstract

<ns3:p>Background Detecting Adverse Drug Events (ADEs) is an emerging research area, attracting great interest in the research community. Better anticipatory management of predisposing factors has considerable potential to improve outcomes. Automatic extraction of ADEs using Natural Language Processing (NLP) has a great potential to significantly facilitate efficient and effective distillation of such knowledge, to better understand and predict risk of adverse events. Methods This systematic review follows the six-stage including the literature from 6 databases (Embase, Medline, Web Of Science Core Collection, ACM Guide to Computing Literature, IEEE Digital Library and Scopus). Following the title, abstract and full-text screenings, characteristics and main findings of the included studies and resources will be tabulated and summarized. The risk of bias and reporting quality was assessed using the PROBAST tool. Results We developed our search strategy and collected all relevant publications. As of December 2024, we have completed all the stages of the systematic review. We identified 178 studies for inclusion through the academic literature search (where data was extracted from all of the papers). Right now, we are writing up the systematic review paper where we are synthesising the different findings. Further refinement of the eligibility criteria and data extraction has been ongoing since August 2022. Conclusion In this systematic review, we will identify and consolidate information and evidence related to the use and effectiveness of existing NLP approaches and tools for automatically detecting ADEs from free text (discharge summaries, General Practitioner notes, social media, etc.). Our findings will improve the understanding of the current landscape of the use of NLP for extracting ADEs. It will lead to better anticipatory management of predisposing factors with the potential to improve outcomes considerably. Our results will also be valuable both to NLP researchers developing methods to extract ADEs and to translational/clinical researchers who use NLP for this purpose and in healthcare in general. For example, from our initial analysis of the studies, we can conclude that the majority of the proposed works are about the detection (extraction) of ADEs from text. An important portion of studies also focus on the binary classification of text (for highlighting if it includes or not ADEs). Different challenges related to the unbalanced dataset, abbreviations and acronyms but also to the lower results with rare ADEs were also mentioned by the studied papers.</ns3:p>
BibTeX
@inproceedings{guellil2024natural,
  author = {Imane Guellil and Jinge Wu and Aryo Pradipta Gema and Farah Francis and Yousra Berrachedi and Nidhaleddine Chenni and Richard Tobin and Clare Llewellyn and Stella Arakelyan and Honghan Wu and Bruce Guthrie and Beatrice Alex},
  title = {{Natural language processing for detecting adverse drug events: A systematic review protocol}},
  booktitle = {NIHR},
  year = {2024},
  doi = {10.3310/nihropenres.13504.3},
}
Figure