Enabling Open Science for Health Research: Collaborative Informatics Environment for Learning on Health Outcomes (CIELO)

Philip Payne, Omkar Lele, Beth Henry Johnson, Erin HolveJournal of Medical Internet Research 2017 · DOI

TL;DR

This paper introduces CIELO, a proof-of-concept open science platform for health research designed to foster collaboration, transparency, and reproducibility. It highlights the critical need for user-centered design and community dialogue to address the unique challenges of health data and build a sustainable open science ecosystem.

What Problem Does This Paper Address?

The paper addresses the growing need for greater accessibility, reproducibility, and rigor in health research. While "Open Science" principles offer a promising path, their practical implementation in the health sciences is complex due to concerns around data privacy, security, and stewardship. The authors highlight the ongoing national dialogue and the challenges in operationalizing open science, exemplified by controversies around research integrity. They aim to explore how a collaborative informatics environment can facilitate open science practices in health research, bridging the gap between theoretical ideals and real-world application.

Methods

The authors implemented a proof-of-concept open science platform called the Collaborative Informatics Environment for Learning on Health Outcomes (CIELO). The development process was user-centered, involving systematic and rigorous user needs assessments with representatives from academic, private, and public sectors. Key conceptual models and functional requirements were informed by these assessments. CIELO was designed to be interoperable with various technologies, adopt an "app store" paradigm for user experience, and incorporate social search and discovery features. The platform utilizes a content and version management system (similar to GitHub), a "folksonomy"-driven annotation and search mechanism, and a simplified user experience leveraging prevailing Web application technologies. Development followed an agile and user-centered design process with a strong emphasis on end-user engagement.

Datasets

Not discussed in the paper.

Key Insights and Ideas

  • User-centered design is paramount for health research platforms: Simply building a technical solution isn't enough; understanding the specific needs and workflows of health researchers is crucial for adoption and success.
  • "App store" paradigm can democratize access: A familiar and intuitive interface, akin to mobile app stores, can make complex data and analytics tools accessible to both technical and non-technical users.
  • Social search and discovery foster collaboration: Enabling users to find and connect with colleagues and their work is essential for building a vibrant research community.
  • Bundling data and code is key: Researchers need flexibility to link multiple versions of code with multiple versions of data, not just a one-to-one mapping, to accurately represent their work.
  • Multi-level sharing permissions are vital: The ability to manage access from private to fully open releases throughout a project's lifecycle is critical for health research.
  • Interoperability is essential for an ecosystem: CIELO aims to be part of a broader "ecosystem" by envisioning interoperability with other platforms like the Open Science Framework (OSF).

Key Results

The public beta testing of CIELO, with nearly 90 registered users from over 20 institutions, provided valuable insights into user needs. Key findings include the importance of flexible data and code bundling, the necessity for multi-level sharing permissions, the benefit of dynamic metadata management, the desirability of cross-linking to external resources, and the value of persistent identifiers like DOIs for attribution and impact. These results highlight the practical requirements for a successful open science platform in health research.

Limitations and Future Work

The authors acknowledge that the CIELO project is a proof-of-concept and that several important questions require further community dialogue and agreement. These include:

  • Incentivizing and sustaining participation: How can current funding, career advancement models, and metrics of scholarly success be adapted to encourage participation in open science platforms?
  • Sustainable fiscal strategy: How can a long-term financial model be developed that aligns with the evolving needs of a high-performing healthcare research community?
  • Platform elasticity and scalability: How can the platform be designed to evolve gracefully over time and avoid becoming obsolete?
  • Interoperability with other platforms: How can CIELO effectively interoperate with other open science initiatives like the OSF to create a comprehensive ecosystem?

Future work will focus on identifying opportunities to deliver reference datasets, developing incentive models, investigating novel methods for addressing privacy and data-sharing constraints, and continuing rigorous user-centered design to identify additional functional requirements. The ultimate goal is to enable the health research community to achieve greater parity with other scientific disciplines that have seen significant productivity gains from open science paradigms.

Summary updated

Abstract

Our initial design and development experience over the course of the CIELO project has provided the basis for a vigorous dialogue between stakeholder community members regarding the capabilities that will add the greatest value to an open science platform for the health research community. A number of important questions around user incentives, sustainability, and scalability will require further community dialogue and agreement.
BibTeX
@article{payne2017enabling,
  author = {Philip Payne and Omkar Lele and Beth Henry Johnson and Erin Holve},
  title = {{Enabling Open Science for Health Research: Collaborative Informatics Environment for Learning on Health Outcomes (CIELO)}},
  journal = {Journal of Medical Internet Research},
  year = {2017},
  doi = {10.2196/jmir.6937},
}
Figure