Lifestyle factors in the biomedical literature: An ontology and comprehensive resources for named entity recognition

Esmaeil Nouranı, Mikaela Koutrouli, Yijia Xie, Danai Vagiaki, Sampo Pyysalo, Katerina Nastou, Søren Brunak, Lars Juhl JensenbioRxiv 2024 · DOI

TL;DR

This paper introduces the first comprehensive ontology (LSFO) and annotated corpus (LSF200) for lifestyle factors (LSFs) in biomedical literature. These resources enable the development of named entity recognition (NER) systems, leading to the identification of over 300 million LSF mentions in scientific texts, paving the way for better understanding of lifestyle's impact on health.

What Problem Does This Paper Address?

Lifestyle factors (LSFs) like diet, exercise, and sleep are increasingly recognized as crucial determinants of individual health, especially for chronic diseases. However, the biomedical literature lacks a systematic way to identify and categorize these factors. This is due to the inherent diversity of LSFs, the absence of a standardized classification, and a lack of dedicated resources like ontologies and annotated datasets for computational analysis. Without these, it's difficult to extract and analyze LSF information at scale, hindering research into their health impacts.

Methods

The authors tackled this problem by developing two key resources and two corresponding NER systems:

  • Lifestyle Factor Ontology (LSFO): A novel, hierarchical ontology was created to classify LSFs. It defines nine main categories (e.g., Nutrition, Socioeconomic factors, Physical activities) and extends to more granular subcategories. This ontology serves as a structured knowledge base for LSFs.
  • LSF200 Corpus: A manually annotated corpus of 200 biomedical abstracts was created, with LSF mentions labeled according to the LSFO. This corpus is crucial for training and evaluating machine learning-based NER systems.
  • Dictionary-based NER System: This system leverages the LSFO to create a dictionary of LSF terms. It then uses a pre-existing, fast dictionary-based NER tool (JensenLab tagger) to identify LSFs in text. This approach prioritizes precision by matching known terms.
  • Transformer-based NER System: An adaptation of a state-of-the-art transformer model (ROBERTa-bio) was trained on the LSF200 corpus. This system learns to identify LSFs based on their context in text, aiming for higher recall.

The dictionary construction involved several expansion stages, starting with manual curation and then incorporating sources like Wikipedia titles, WordNet, Wikidata, and other ontologies, with manual validation at each step.

Datasets

  • LSF200 Corpus: A newly introduced, manually annotated corpus of 200 biomedical abstracts. It contains 1876 annotated LSF mentions across nine categories, plus an "Out of Context" (OOC) category to flag mentions that don't fit the LSF definition in a given context.
  • PubMed Abstracts and PMC Open Access Articles: Large-scale datasets (36.1 million PubMed abstracts and 4.5 million PMC OA articles) were used to apply the developed NER systems and identify millions of LSF mentions.

Key Insights and Ideas

  • The "Out of Context" (OOC) challenge: Identifying LSFs is complicated by words that can be LSFs in one context but not another (e.g., "tobacco" as a substance use vs. "tobacco company"). The LSF200 corpus includes OOC annotations to help address this, and the authors found that handling OOC mentions significantly impacts the precision of dictionary-based NER.
  • Ontology quality matters: The authors found that only a small fraction of false negatives (missing LSF mentions) were due to missing concepts in the LSFO itself. This suggests the ontology has good breadth, and most missing mentions are due to fine-grained concepts or synonyms not yet captured.
  • Precision vs. Recall Trade-off: The dictionary-based NER system achieves high precision but lower recall, while the transformer-based system offers better recall at the cost of slightly lower precision. This highlights the inherent trade-off between these two approaches.
  • Transformer models excel at recall: The transformer-based NER system showed significant improvements in recall across most LSF categories, demonstrating its ability to generalize and identify LSFs even when they are not explicitly in a predefined dictionary.

Key Results

  • LSFO and LSF200: The creation of a comprehensive LSF ontology and a manually annotated corpus are significant contributions to the field.
  • NER Performance:
    • Dictionary-based NER: Achieved an F-score of 64% on the LSF200 corpus.
  • Transformer-based NER: Achieved an F-score of 76% on the LSF200 corpus.
  • Large-scale application: Over 300 million LSF mentions were identified in PubMed and PMC OA articles using both systems.
  • Error Analysis: The dictionary-based system's main errors were false negatives (missed mentions) due to abbreviations/synonyms and ambiguous names. The transformer-based system's errors were more often "model errors" or annotation errors.

Limitations and Future Work

  • Dictionary-based NER Recall: The dictionary-based system's low recall for certain categories (e.g., "Environmental exposures") could be improved by integrating more fine-grained concepts from external resources via the established Xrefs.
  • Transformer-based NER Performance: The transformer system performed worse than the dictionary-based one for "Substance use" and "Sleep" categories, suggesting areas for further model refinement.
  • Ambiguous Names: The authors intentionally excluded some ambiguous names to maintain precision in the dictionary-based system, leading to potential false negatives. Future work could explore strategies to handle ambiguity more effectively.
  • Future Directions: The authors suggest that the developed resources (LSFO, LSF200, and the NER systems) are publicly available and can serve as a foundation for further research, including relation extraction and a deeper understanding of lifestyle's role in health.

Summary updated

Abstract

Abstract Motivation Despite lifestyle factors (LSFs) being increasingly acknowledged in shaping individual health trajectories, particularly in chronic diseases, they have still not been systematically described in the biomedical literature. This is in part because no named entity recognition (NER) system exists, which can comprehensively detect all types of LSFs in text. The task is challenging due to their inherent diversity, lack of a comprehensive LSF classification for dictionary-based NER, and lack of a corpus for deep learning-based NER. Results We present a novel Lifestyle Factor Ontology (LSFO), which we used to develop a dictionarybased system for recognition and normalization of LSFs. Additionally, we introduce a manually annotated corpus for LSFs (LSF200) suitable for training and evaluation of NER systems, and use it to train a transformer-based system. Evaluating the performance of both NER systems on the corpus revealed an F-score of 64% for the dictionary-based system and 76% for the transformer-based system. Largescale application of these systems on PubMed abstracts and PMC Open Access articles identified over 300 million mentions of LSF in the biomedical literature. Availability LSFO, the annotated LSF200 corpus, and the detected LSFs in PubMed and PMC-OA articles using both NER systems, are available under open licenses via the following GitHub repository: https://github.com/EsmaeilNourani/LSFO-expansion . This repository contains links to two associated GitHub repositories and a Zenodo project related to the study. LSFO is also available at BioPortal: https://bioportal.bioontology.org/ontologies/LSFO . Contact [email protected] , [email protected] , [email protected] Supplementary information Supplementary data are available at Bioinformatics online.
BibTeX
@misc{nouran2024lifestyle,
  author = {Esmaeil Nouranı and Mikaela Koutrouli and Yijia Xie and Danai Vagiaki and Sampo Pyysalo and Katerina Nastou and Søren Brunak and Lars Juhl Jensen},
  title = {{Lifestyle factors in the biomedical literature: An ontology and comprehensive resources for named entity recognition}},
  year = {2024},
  doi = {10.1101/2024.06.13.598816},
  howpublished = {bioRxiv},
}
Figure