Prototyping a Biomedical Ontology Recommender Service

Clément Jonquet, Nigam H. Shah, Mark A. MusenSPIRE 2009

TL;DR

This paper introduces a novel service that automatically recommends relevant biomedical ontologies for annotating datasets. By leveraging existing annotation tools and scoring ontologies based on their concept coverage of dataset descriptions, it tackles the challenge of selecting appropriate ontologies from a vast and growing landscape, saving researchers significant time and effort.

What Problem Does This Paper Address?

Biomedical researchers increasingly rely on ontologies to structure, annotate, and integrate data across diverse repositories. However, the sheer volume and variety of available biomedical ontologies make it a daunting task for scientists to identify the most suitable ones for their specific annotation needs. This often leads to researchers either missing crucial ontologies, having to re-annotate datasets, or even developing new ontologies unnecessarily. The paper addresses this "ontology selection problem" by proposing an automated solution.

Methods

The core of the proposed solution is an ontology recommender service. This service operates in two main steps:

  1. Annotation Generation: It utilizes the Open Biomedical Annotator (OBA), a web service developed by the National Center for Biomedical Ontology (NCBO). OBA takes textual metadata from a dataset and annotates it with biomedical ontology concepts. This annotation process involves:
  2. Direct Annotation: Matching terms in the text directly with concept names or synonyms from ontologies.
  3. Semantic Expansion: Leveraging ontology relationships (like "is-a" hierarchies and mappings) to infer additional annotations. The OBA service can draw from a large collection of ontologies, including those in the UMLS Metathesaurus and the NCBO BioPortal.

  4. Ontology Scoring and Recommendation: The recommender service then aggregates the annotations generated by OBA. It scores each ontology based on the sum of the scores of the annotations that used concepts from that particular ontology. The scoring algorithm assigns different weights to annotations based on their context (e.g., direct match vs. synonym match, preferred name vs. ancestor term) and the type of expansion used. This weighted scoring ensures that more reliable and relevant annotations contribute more to an ontology's overall score. Finally, the service presents a ranked list of ontologies, with the highest-scoring ones being the most recommended.

Datasets

The paper does not explicitly list specific datasets used for training the recommender service. Instead, it describes using three types of example datasets for evaluation and demonstration purposes:

  • PubMed Articles: Top 10 articles retrieved using the query "melanoma AND skin disease," with concatenated title and abstract fields.
  • Clinical Trials: Top 10 Clinicaltrials.gov trials (NCT) retrieved using the query "melanoma AND skin disease," with concatenated title, purpose, condition, and intervention fields.
  • High Throughput Experiments: Top 10 Gene Expression Omnibus datasets (GDS) retrieved using the query "melanoma," with concatenated title and summary fields.

These examples highlight how the recommender service can be applied to different types of biomedical data.

Key Insights and Ideas

  • Automating a Manual Burden: The most significant insight is the successful automation of a time-consuming and often expert-dependent task – selecting relevant ontologies.
  • Leveraging Existing Tools: The paper demonstrates a clever way to repurpose an existing annotation service (OBA) for a new purpose (ontology recommendation).
  • Context-Aware Scoring is Crucial: The weighted scoring mechanism, which considers the context of an annotation (e.g., direct match vs. inferred match), is vital for accurately ranking ontologies. This shows that not all annotations are created equal.
  • Big Ontologies Often Score High (But Not Always): Larger ontologies like NCI Thesaurus tend to receive high scores due to their extensive concept coverage. However, the paper also notes that smaller, more specialized ontologies can appear high in the rankings if they are particularly relevant to the input text, underscoring the importance of the scoring method.
  • Domain-Specific Relevance: The results show that certain ontologies are recommended for specific data types (e.g., anatomy ontologies for clinical trials), demonstrating the service's ability to identify domain-specific relevance.

Key Results

The paper presents results from applying the recommender service to the three types of datasets mentioned above. While specific quantitative improvements (e.g., percentage of time saved) are not detailed, the tables show the top-ranked ontologies for each dataset type.

  • For PubMed articles, NCI Thesaurus was the highest-scoring ontology, followed by Human disease.
  • For Clinical Trials, NCI Thesaurus again ranked highest, with Human developmental anatomy, timed version also appearing prominently.
  • For GEO series, NCI Thesaurus and Human disease were top contenders, with Galen and Experimental Factor Ontology also scoring well.

These results illustrate that the recommender service successfully identifies a set of relevant ontologies for each data type, with some ontologies appearing consistently across different domains.

Limitations and Future Work

The authors acknowledge several limitations and propose future directions:

  • Evaluation with Experts: The current evaluation is based on example datasets. The authors are in the process of evaluating the recommender service with biomedical experts and curators to assess its utility in real-world scenarios.
  • Scoring Method Refinements: Future work includes investigating different scoring methods to support various recommendation scenarios, such as recommending very specific, small ontologies. This might involve incorporating ontology size into the scoring.
  • User Customization: Allowing users to customize scoring methods by adjusting weights for different contexts is another envisioned improvement.
  • Integration: The service is deployed as a REST web service, facilitating its integration into automatic workflows, but further embedding and usability enhancements are implied.

Summary updated

Abstract

International audience
BibTeX
@inproceedings{jonquet2009prototyping,
  author = {Clément Jonquet and Nigam H. Shah and Mark A. Musen},
  title = {{Prototyping a Biomedical Ontology Recommender Service}},
  booktitle = {SPIRE},
  year = {2009},
}
Figure