Extraction of Lactation Frames from Drug Labels and LactMed

Heath Goodrum, Meghana Gudala, Ankita Misra, Kirk RobertsACL 2019

TL;DR

This paper introduces a novel Natural Language Processing (NLP) approach to automatically extract critical information about drug safety during lactation from two distinct sources: FDA-mandated drug labels and the NLM's LactMed database. By employing a frame semantic model, the research aims to identify and compare information about drug excretion, infant absorption, and adverse reactions, ultimately paving the way for more consistent and reliable health guidance for nursing mothers.

What Problem Does This Paper Address?

Providing accurate and consistent health information for nursing mothers is a significant challenge. Randomized controlled trials, the gold standard for drug evaluation, often exclude pregnant and breastfeeding women, leading to a scarcity of high-quality evidence regarding drug safety during lactation. This data gap can result in misinformation and discourage mothers from taking necessary medications. While drug labels and specialized databases like LactMed aim to fill this void, their unstructured nature and varying formats make it difficult to synchronize and compare information, potentially leading to discrepancies and outdated guidance. This paper tackles the problem of automatically extracting and comparing lactation-specific drug information from these disparate sources to bridge this knowledge gap.

Methods

The core of this research lies in a frame semantic approach to information extraction. The authors define seven key "frames" representing crucial aspects of lactation-related drug information: INFORMATION_AVAILABILITY, EFFECT_ON_MILK_SUPPLY, EXCRETION_INTO_MILK, ABSORPTION, ADVERSE_REACTION, ALTERNATIVES, and VERDICT. These frames, along with their specific elements (e.g., QUANTITY, REACTION, DECISION), provide a structured way to represent the information.

The NLP system employs a standard bi-directional Long Short-Term Memory (Bi-LSTM) with a Conditional Random Field (CRF) decoder. This architecture is designed to handle sequential data like text. The system operates in two steps: first, identifying lexical units that trigger the defined frames (akin to named entity recognition), and second, extracting the frame elements associated with these identified frames (relation extraction). The system leverages both character and word embeddings to represent the text, experimenting with various pre-trained and custom-built embeddings.

Datasets

The study utilizes two primary datasets:

  • Drug Labels: Extracted from the DailyMed database, which contains FDA-mandated drug labels. The "Lactation" section (or "Nursing Mothers" for older labels) was specifically targeted. After de-duplication based on drug name, 4,486 unique drug label documents were created.
  • LactMed: The "Summary of Use During Lactation" section from the LactMed database, a government-maintained resource. This resulted in 1,151 unique documents.

A combined corpus of 900 documents (450 drug labels and 450 LactMed entries) was manually annotated for training and evaluation.

Key Insights and Ideas

  • Frame Semantics as a Unifying Framework: The paper demonstrates the effectiveness of frame semantics in structuring and extracting complex, nuanced information from diverse biomedical text sources. This approach provides a standardized way to represent information that might otherwise be expressed in highly varied language.
  • Cross-Corpus Transferability is Challenging but Possible: While drug labels and LactMed entries have distinct characteristics, training a model on a combined dataset significantly improves performance compared to training on either source alone. This suggests underlying similarities that can be leveraged, though domain adaptation techniques might further enhance performance.
  • The Importance of Contextual Embeddings: The choice of word embeddings significantly impacts performance. While pre-trained open-domain embeddings showed promise, custom embeddings trained on the specific lactation data (drug labels and LactMed combined) also yielded strong results, highlighting the value of domain-specific representations.
  • Annotation Quality is Paramount: The learning curve suggests that while more data can improve performance, the authors note that many "errors" might actually be legitimate frame instances due to inconsistencies in manual annotation. This underscores the critical need for rigorous annotation guidelines and quality control.

Key Results

  • The best performing model, using MIMIC + GloVe (Wikipedia) embeddings, achieved an F1 score of 83.89 for frames and 70.93 for frame elements on the combined DL+LM dataset.
  • Training on a combined dataset of drug labels and LactMed improved performance on the LactMed test set (F1 of 77.18) compared to training solely on LactMed (F1 of 71.54).
  • The most frequently identified frames were EXCRETION_INTO_MILK (853 instances), VERDICT (852 instances), and ADVERSE_REACTION (727 instances).
  • Inter-annotator agreement, measured by F1, was generally good, with specific frames like INFORMATION_AVAILABILITY and VERDICT showing high agreement.

Limitations and Future Work

  • Annotation Inconsistencies: The authors acknowledge that inconsistencies in manual annotation can lead to errors and that further quality control on existing annotations is a priority.
  • Domain Adaptation: While combined training improves performance, the differences between drug labels and LactMed suggest that domain adaptation methods could further enhance the system's ability to leverage larger datasets.
  • Comparing Frames: This paper focuses on extracting individual frame instances. Future work will involve developing methods to compare these extracted frames across different sources to automatically detect inconsistencies, conflicting information, and information gaps. This comparison will likely require sophisticated techniques beyond simple rule-based matching.
  • Generalizability: The authors posit that the frame-based approach is generalizable to other lactation information sources and potentially to other pharmaceutical sub-populations, though this would require significant annotation effort.

Summary updated

BibTeX
@inproceedings{goodrum2019extraction,
  author = {Heath Goodrum and Meghana Gudala and Ankita Misra and Kirk Roberts},
  title = {{Extraction of Lactation Frames from Drug Labels and LactMed}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
}
Figure