An Online Repository for Pre-Clinical Imaging Protocols (PIPs)

Seth T. Gammon, Allison S. Cohen, Adrienne L. Lehnert, Daniel C. Sullivan, Dariya Malyarenko, Henry Charles Manning, David A. Hormuth, Heike E. Daldrup‐Link, Hongyu An, James D. Quirk, Kooresh I. Shoghi, Mark D. Pagel, Paul E. Kinahan, Robert S. Miyaoka, A. McGarry Houghton, Michael T. Lewis, Peder E. Z. Larson, Renuka Sriram, Stephanie J. Blocker, Stephen Pickup, Alexandra Badea, Cristian T. Badea, Thomas E. Yankeelov, Thomas L. ChenevertTomography 2023 · DOI

TL;DR

This paper introduces a standardized online repository for pre-clinical imaging protocols (PIPs) to address the growing reproducibility crisis in scientific research. By providing accessible, detailed, and citable protocol templates for PET, CT, and MRI, the authors aim to improve the transparency and repeatability of pre-clinical imaging studies.

What Problem Does This Paper Address?

The scientific community faces a significant "reproducibility crisis," where many published studies cannot be independently replicated. While some fields have addressed this with specialized journals for detailed protocols, this format is often ill-suited for the complex needs of quantitative imaging, which requires extensive details on instrument validation, imaging parameters, and statistical analysis. Traditional journal articles offer insufficient space for these crucial details, and supplemental materials are often not updated or easily discoverable. This lack of detailed, accessible, and updatable methodology hinders the ability of other researchers to reproduce findings, slowing scientific progress and potentially leading to wasted resources.

Methods

The authors developed standardized protocol templates for pre-clinical imaging modalities including Positron Emission Tomography (PET), X-ray Computed Tomography (CT), and Magnetic Resonance Imaging (MRI). These templates are designed to capture critical modality-specific information in a harmonized layout. Key features include:

  • Executive Summary: A brief overview of the protocol's goal and outcome.
  • Pre-Clinical Imaging Claim: A section for linking to peer-reviewed publications or pre-prints that validate the protocol's precision and accuracy.
  • Imaging Process Specifications: Detailed fields for scanner hardware, acquisition techniques (including anesthesia, contrast agents, and scanning parameters), animal models, and quantitative metrics.
  • Protocols.io Platform: The templates and resulting protocols are hosted on the publicly accessible and searchable platform, protocols.io. This allows for easy access, version control, authorship tracking, and community feedback.
  • Iterative Development: The templates were developed through a collaborative process involving sub-committees of experts from multiple institutions, with iterative review and feedback.

Datasets

Not discussed in the paper. The focus is on the creation of a framework for protocols, not on specific datasets used for training or evaluation.

Key Insights and Ideas

  • The "Reproducibility Crisis" is Not Limited to Complex Experiments: The paper highlights that reproducibility issues are prevalent across various scientific disciplines, not just those involving intricate physical setups.
  • Detailed Methods are Crucial for Reproducibility: The authors emphasize that the inability to reach original research teams is a strong predictor of failed reproductions, underscoring the need for comprehensive methodological documentation.
  • Existing Journal Formats are Insufficient: Traditional journal articles and even supplemental materials lack the structure and permanence needed for detailed, evolving imaging protocols.
  • Online Repositories Offer a Solution: Platforms like protocols.io provide a flexible, accessible, and citable space for researchers to share and update detailed protocols, overcoming the limitations of print-based publications.
  • Standardization Enhances Usability: Creating modality-specific templates ensures a consistent structure, making it easier for researchers to write and find protocols across different imaging techniques.

Key Results

The paper presents the development and proposed implementation of standardized protocol templates for pre-clinical PET, CT, and MRI. These templates are now available on protocols.io. The authors anticipate that these PIPs will:

  • Improve the reproducibility of pre-clinical imaging studies.
  • Facilitate the sharing of detailed methods and instrument validation.
  • Enable researchers to more easily adapt and build upon existing protocols.
  • Provide a sustainable historical record of imaging methodologies.
  • Potentially accelerate clinical translation by providing robust pre-clinical data.

Limitations and Future Work

  • Concentration Risk: The reliance on protocols.io presents a potential risk if the platform changes its terms or becomes unavailable. GitHub is suggested as a backup.
  • Philosophical Risk: The paper acknowledges the possibility that some pre-clinical imaging experiments may be inherently irreproducible due to factors like vendor variability, which PIPs might help highlight.
  • Adoption and Utility: The long-term success of the PIPs will depend on their adoption and perceived utility by the research community.
  • Future Directions:
    • Incorporating feedback from early adopters to refine templates.
  • Developing templates for other modalities like MRS, SPECT, and optical imaging.
  • Facilitating language localization of templates and protocols through international collaborations.
  • Creating a public database of PIPs with a user-friendly interface.
  • Harmonizing PIP data with pre-clinical DICOM formats for automated metadata capture.

Summary updated

Abstract

Providing method descriptions that are more detailed than currently available in typical peer reviewed journals has been identified as an actionable area for improvement. In the biochemical and cell biology space, this need has been met through the creation of new journals focused on detailed protocols and materials sourcing. However, this format is not well suited for capturing instrument validation, detailed imaging protocols, and extensive statistical analysis. Furthermore, the need for additional information must be counterbalanced by the additional time burden placed upon researchers who may be already overtasked. To address these competing issues, this white paper describes protocol templates for positron emission tomography (PET), X-ray computed tomography (CT), and magnetic resonance imaging (MRI) that can be leveraged by the broad community of quantitative imaging experts to write and self-publish protocols in protocols.io. Similar to the Structured Transparent Accessible Reproducible (STAR) or Journal of Visualized Experiments (JoVE) articles, authors are encouraged to publish peer reviewed papers and then to submit more detailed experimental protocols using this template to the online resource. Such protocols should be easy to use, readily accessible, readily searchable, considered open access, enable community feedback, editable, and citable by the author.
BibTeX
@inproceedings{gammon2023online,
  author = {Seth T. Gammon and Allison S. Cohen and Adrienne L. Lehnert and Daniel C. Sullivan and Dariya Malyarenko and Henry Charles Manning and David A. Hormuth and Heike E. Daldrup‐Link and Hongyu An and James D. Quirk and Kooresh I. Shoghi and Mark D. Pagel and Paul E. Kinahan and Robert S. Miyaoka and A. McGarry Houghton and Michael T. Lewis and Peder E. Z. Larson and Renuka Sriram and Stephanie J. Blocker and Stephen Pickup and Alexandra Badea and Cristian T. Badea and Thomas E. Yankeelov and Thomas L. Chenevert},
  title = {{An Online Repository for Pre-Clinical Imaging Protocols (PIPs)}},
  booktitle = {Tomography},
  year = {2023},
  doi = {10.3390/tomography9020060},
}
Figure