TL;DR
The paper surveys existing ontologies for representing scientific assertions, evidence, and provenance, highlighting a critical gap in standardization for assertion and evidence ontologies. This work is crucial for improving scientific reproducibility and knowledge discovery in an era of rapidly expanding research output.
What Problem Does This Paper Address?
The scientific landscape is exploding with new publications and complex experimental methods. This rapid growth makes it incredibly challenging for researchers to keep up with the latest advancements and, more importantly, to reproduce findings. A significant hurdle is the "reproducibility crisis," where a large percentage of researchers struggle to replicate experiments. This difficulty stems from a lack of detailed documentation regarding the evidence supporting scientific claims (assertions) and the step-by-step processes (provenance) that led to those claims. Without this structured information, scientific progress is hampered, and investments in research are less fruitful. The paper aims to address this by surveying existing tools (ontologies) that could help formalize and standardize this crucial information.
Methods
The authors conducted a systematic survey of ontologies related to scientific assertions, evidence, and provenance. Their methodology involved:
- Defining Search Criteria: They established keywords to find relevant ontologies.
- Performing Searches: They utilized academic search engines (Google Scholar, IEEE Xplore) and specialized ontology portals (BioPortal, OntoPortal), along with general search engines (Google, GitHub).
- Reviewing Search Results: They evaluated the relevance of retrieved ontologies by checking if they captured essential elements like assertions, evidence, and provenance in the context of scientific publications and experiments. Ontologies focusing on other domains or aspects were excluded.
- Refining Search: Criteria were adjusted (e.g., publication date, keyword variations) to improve search outcomes.
- Finalizing Results: The selected ontologies were analyzed.
- Performing Analysis: This included examining the number of classes, properties, reused components, and the domains of application for each ontology.
Datasets
Not discussed in the paper. The paper focuses on surveying existing ontologies rather than training or evaluating models on specific datasets.
Key Insights and Ideas
- The "Reproducibility Crisis" is Real and Costly: The paper underscores that a significant portion of researchers cannot reproduce experiments, leading to substantial financial waste.
- Ontologies are Key to FAIR Science: Semantic web technologies like ontologies and knowledge graphs offer a structured way to represent scientific data, aligning with FAIR principles (Findable, Accessible, Interoperable, Reusable) and facilitating knowledge discovery.
- Provenance is Relatively Standardized, Assertions/Evidence Aren't: While the PROV ontology provides a widely adopted standard for provenance, there's a notable lack of standardization for ontologies that capture scientific assertions and their supporting evidence.
- Ontology Discovery is Challenging: Finding and accessing relevant ontologies can be difficult due to hosting heterogeneity and inconsistent documentation.
- Domain-Specific vs. General Ontologies: Many ontologies are tailored to specific domains (like life sciences or neuroimaging), while others aim for broader applicability. Domain-agnostic ontologies like PROV are often reused as foundational models.
Key Results
The survey identified:
- 13 ontologies focused on assertions and evidence.
- 10 ontologies focused on provenance.
The paper details the characteristics of these ontologies, including the number of classes, properties, their language (mostly OWL), and the domains they cover. It highlights that while some ontologies explicitly model "assertion" and "evidence" (e.g., SEPIO, RDO, SIO), others use broader terms like "Argument" or "Claim" and rely on provenance for evidence. For provenance, PROV is a dominant, standardized model, with many domain-specific ontologies reusing it.
Limitations and Future Work
Limitations:
- Accessibility and Findability: Many ontologies were difficult to access or find, hindering their reusability.
- Lack of Vocabulary Integration: Even when standards exist (like PROV), they are not always fully leveraged by domain-specific ontologies.
- Lack of Standardization for Assertions/Evidence: This is the most significant gap identified, contrasting with the standardization seen in provenance.
Future Work:
- Develop a Standardized Ontology for Assertions and Evidence: Similar to PROV for provenance, a general, standardized ontology is needed for assertions and evidence.
- Address Sustainability and Accessibility: Efforts should focus on making ontologies more accessible and sustainable.
- Promote Vocabulary Integration: Encourage the consistent use of existing standards.
- Develop Tools: Create tools that leverage and reuse formalized ontologies to support scientific knowledge representation and reproducibility.
- Analyze Ontology Usage: Further research into ontology adoption rates and trends is needed.
Summary updated