Recommending Concepts to Experts: An Exploration of Recommender Techniques for Collaborative Ontology Engineering Platforms in the Biomedical Domain

Simon Walk, Markus Strohmaier, Tania Tudorache, Natalya Fridman Noy, Csongor Nyulas, Mark A. MusenICBO 2012 · OpenReview

Abstract

Large-scale collaborative ontology engineering, such as the development of the ICD-11, faces significant coordination challenges as hundreds of experts navigate tens of thousands of concepts. This paper investigates the application of recommender systems to assist experts in identifying relevant areas for contribution. By comparing content-based, collaborative filtering, and knowledge-based approaches using real-world data from the iCAT platform, the study demonstrates that these techniques provide complementary perspectives that can effectively guide expert participation and mitigate information overload in complex knowledge projects.

Introduction: The Coordination Challenge in Ontology Engineering

Building a comprehensive ontology—a formal map of medical knowledge—is a monumental task. Historically, projects like the International Classification of Diseases (ICD) were managed by small, centralized groups. However, the 11th revision (ICD-11) transitioned to a collaborative, "crowdsourcing" model involving hundreds of international experts.

While this approach increases coverage and quality, it introduces a massive coordination problem. Experts often struggle to find where their specific knowledge is most needed among tens of thousands of concepts. This leads to low participation, overlapping efforts, and "orphaned" sections of the ontology. This research explores whether recommender systems—similar to those used by consumer platforms like Netflix—can effectively suggest medical concepts to experts to bridge this gap.

The iCAT Platform and Dataset

The study utilized real-world interaction logs from the iCAT (Collaborative Authoring Tool), the primary platform used to build the ICD-11. The dataset spanned from November 2009 to August 2011, capturing the complex interactions of experts within the hierarchical structure of the ontology.

The iCAT Collaborative Authoring Tool interface showing the hierarchical structure of ICD categories and concept details.

The researchers focused on anonymized expert profiles (labeled LB, AR, and RC) to demonstrate how different recommendation algorithms behave in a live environment.

Recommendation Methodologies

The researchers implemented and compared three distinct recommendation strategies:

  1. Content-Based Filtering: This approach analyzes the "textual DNA" of concepts. Using cosine similarity, it compares the linguistic content (titles and definitions) of concepts an expert has previously edited with other concepts in the system.
  2. Collaborative Filtering: This method focuses on "social DNA" by identifying users with similar editing patterns. Using the Jaccard coefficient, the system recommends concepts to a user that have been edited by their "peers" with similar interests.
  3. Knowledge-Based Recommenders: This strategy leverages the unique structural properties of an ontology. It employs graph traversal to find concepts that are logically related through "is-a" or other semantic relationships, even if they lack linguistic overlap or shared user history.
A graph-based representation of logical relationships between medical codes, illustrating the paths used by knowledge-based recommenders.

Key Insights and Results

The most significant finding is that these three methods are complementary rather than redundant. They provide different "lenses" on the data:

  • Content-based methods are highly specific, suggesting granular concepts (e.g., specific types of skin abscesses).
  • Collaborative filtering tends to suggest broader, high-level categories, reflecting the general domain expertise of peer groups.
  • Knowledge-based methods identify logical neighbors that are medically related but might be linguistically distinct.

The proof-of-concept demonstrated technical feasibility. For instance, in a ranked comparison for a sample user (RC), collaborative filtering produced high similarity scores, suggesting it is effective at identifying broad areas of interest. Meanwhile, content-based methods provided lower but highly targeted results, which are particularly valuable for solving the "cold start" problem when user interaction history is limited.

Conclusion and Future Work

While the algorithms produced relevant lists, the primary limitation of this study is the lack of formal user evaluation with medical experts. Future research will focus on developing hybrid "meta-recommender" systems, incorporating real-time browsing data, and conducting live trials within the iCAT platform to measure the actual impact on expert productivity and ontology quality.

Summary updated

BibTeX
@inproceedings{walk2012recommending,
  author = {Simon Walk and Markus Strohmaier and Tania Tudorache and Natalya Fridman Noy and Csongor Nyulas and Mark A. Musen},
  title = {{Recommending Concepts to Experts: An Exploration of Recommender Techniques for Collaborative Ontology Engineering Platforms in the Biomedical Domain}},
  booktitle = {ICBO},
  year = {2012},
  url = {https://openreview.net/forum?id=YWqlFkzWUh},
}
Figure