Information Retrieval using Machine Learning for Biomarker Curation in the Exposome-Explorer

André Lamúrias, Sofia Jesus, Vanessa Neveu, Reza M. Salek, Francisco M. Couto

TL;DR

This paper introduces a machine learning approach to automate the tedious process of curating scientific literature for the Exposome-Explorer database, which tracks biomarkers of environmental exposure. By training classifiers on existing data, they significantly reduced the manual screening effort by nearly 90% while identifying new candidate biomarkers.

What Problem Does This Paper Address?

The Exposome-Explorer is a crucial database for understanding how environmental factors impact health, but its creation and maintenance are incredibly labor-intensive. Manually sifting through millions of scientific articles to identify relevant biomarkers requires specialized expertise and vast amounts of time. This manual curation process is a bottleneck, hindering the database's growth and the timely integration of new research. The paper aims to alleviate this burden by developing a machine learning system to assist in the initial screening of literature, effectively acting as a "triage" system.

Methods

The core of this work is a supervised machine learning approach for document classification. The researchers experimented with various algorithms, including Logistic Regression, Naïve Bayes, Neural Networks, Random Forests, and Support Vector Machines. They explored different combinations of article features (titles, abstracts, and metadata) for training these models. Key preprocessing steps involved natural language processing techniques like stemming and stop-word removal, followed by transforming textual data into numerical representations suitable for machine learning. They also investigated ensemble methods (Bagging and Stacking) to potentially improve performance by combining multiple classifiers. Performance was evaluated using metrics like Precision, Recall, F1-score, F2-score, and ROC AUC.

Datasets

The primary dataset used was derived from the existing Exposome-Explorer database, which was initially populated by a manual literature search yielding over 8500 citations. From these, 480 publications were used to extract biomarker information. For training and testing their machine learning models, the researchers expanded this dataset by retrieving titles, abstracts, and metadata for 7083 publications using PubMed. This expanded dataset was then labeled as "relevant" (1) or "irrelevant" (0) based on whether the publication was used in the original database curation. A subset focusing on "dietary biomarkers" (3016 publications) was also used for specific experiments.

Key Insights and Ideas

  • Titles and Abstracts are Key: The study found that combining titles and abstracts generally yielded better classification results than using either alone or just titles and metadata. This highlights the importance of both concise summaries and detailed descriptions for identifying relevant research.
  • Logistic Regression Shines: The Logistic Regression algorithm, particularly when trained on titles and abstracts, emerged as the top performer, achieving a high F2-score. This suggests that for this specific task, a simpler, interpretable model can be highly effective.
  • Reducing Manual Effort is Achievable: The proposed methodology demonstrated a significant reduction in the number of articles requiring manual screening, by nearly 90%. This is a crucial practical outcome for resource-constrained curation efforts.
  • Automated Biomarker Discovery: Beyond just classifying articles, the system was able to identify candidate biomarkers within the relevant abstracts, contributing to the discovery of new entries for the database.

Key Results

The top-performing model, a Logistic Regression classifier trained on titles and abstracts, achieved an F2-score of 70.1%. This model successfully reduced the manual screening workload by approximately 90%, while only misclassifying about 22.1% of relevant articles. The system identified 545 candidate biomarkers from the classified relevant articles, with 460 of these being new entries for the Exposome-Explorer database.

Limitations and Future Work

The authors acknowledge that while the models performed well on dietary biomarkers, their performance on other classes of biomarkers (like pollutants) was lower. They suggest that this difference might be due to variations in the nature of the data and the attentiveness of manual curation for different biomarker types. Future work includes:

  • Improving models that use metadata, potentially by assigning weights to authors or creating new features from author combinations.
  • Investigating the impact of recognized biomarkers on the classification process itself, and exploring ways to mitigate over-fitting by using broader categories (e.g., "chemical" instead of specific chemical tokens).
  • Expanding the gold standard dataset to include more types of biomarkers beyond dietary ones.
  • Exploring deep learning models, which might require larger datasets, and applying them to other biomedical corpora.

Summary updated

Abstract

Abstract In 2016, the International Agency for Research on Cancer, part of the World Health Organization, released the Exposome-Explorer, the first database dedicated to biomarkers of exposure for environmental risk factors for diseases. The database contents resulted from a manual literature search that yielded over 8500 citations, but only a small fraction of these publications were used in the final database. Manually curating a database is time-consuming and requires domain expertise to gather relevant data scattered throughout millions of articles. This work proposes a supervised machine learning approach to assist the previous manual literature retrieval process. The manually retrieved corpus of scientific publications used in the Exposome-Explorer was used as training and testing sets for the machine learning models (classifiers). Several parameters and algorithms were evaluated to predict an article’s relevance based on different datasets made of titles, abstracts and metadata. The top performance classifier was built with the Logistic Regression algorithm using the title and abstract set, achieving an F2-score of 70.1%. Furthermore, from 705 articles classified as relevant, we extracted 545 biomarkers, including 460 new candidate entries to the Exposome-Explorer database. Our methodology reduced the number of articles to be manually screened by the database curators by nearly 90%, while only misclassifying 22.1% of the relevant articles. We expect that this methodology can also be applied to similar biomarkers datasets or be adapted to assist the manual curation process of similar chemical or disease databases.
BibTeX
@article{lamurias2021information,
  author = {André Lamúrias and Sofia Jesus and Vanessa Neveu and Reza M. Salek and Francisco M. Couto},
  title = {{Information Retrieval using Machine Learning for Biomarker Curation in the Exposome-Explorer}},
  journal = {Frontiers Res. Metrics Anal.},
  year = {2021},
  doi = {10.3389/frma.2021.689264},
}
Figure