TL;DR
This paper introduces a standardized online repository for pre-clinical imaging protocols (PIPs) to address the growing reproducibility crisis in scientific research. By providing accessible, detailed, and citable protocol templates for PET, CT, and MRI, the authors aim to improve the transparency and repeatability of pre-clinical imaging studies.
What Problem Does This Paper Address?
The scientific community faces a significant "reproducibility crisis," where many published studies cannot be independently replicated. While some fields have addressed this with specialized journals for detailed protocols, this format is often ill-suited for the complex needs of quantitative imaging, which requires extensive details on instrument validation, imaging parameters, and statistical analysis. Traditional journal articles offer insufficient space for these crucial details, and supplemental materials are often not updated or easily discoverable. This lack of detailed, accessible, and updatable methodology hinders the ability of other researchers to reproduce findings, slowing scientific progress and potentially leading to wasted resources.
Methods
The authors developed standardized protocol templates for pre-clinical imaging modalities including Positron Emission Tomography (PET), X-ray Computed Tomography (CT), and Magnetic Resonance Imaging (MRI). These templates are designed to capture critical modality-specific information in a harmonized layout. Key features include:
- Executive Summary: A brief overview of the protocol's goal and outcome.
- Pre-Clinical Imaging Claim: A section for linking to peer-reviewed publications or pre-prints that validate the protocol's precision and accuracy.
- Imaging Process Specifications: Detailed fields for scanner hardware, acquisition techniques (including anesthesia, contrast agents, and scanning parameters), animal models, and quantitative metrics.
- Protocols.io Platform: The templates and resulting protocols are hosted on the publicly accessible and searchable platform, protocols.io. This allows for easy access, version control, authorship tracking, and community feedback.
- Iterative Development: The templates were developed through a collaborative process involving sub-committees of experts from multiple institutions, with iterative review and feedback.
Datasets
Not discussed in the paper. The focus is on the creation of a framework for protocols, not on specific datasets used for training or evaluation.
Key Insights and Ideas
- The "Reproducibility Crisis" is Not Limited to Complex Experiments: The paper highlights that reproducibility issues are prevalent across various scientific disciplines, not just those involving intricate physical setups.
- Detailed Methods are Crucial for Reproducibility: The authors emphasize that the inability to reach original research teams is a strong predictor of failed reproductions, underscoring the need for comprehensive methodological documentation.
- Existing Journal Formats are Insufficient: Traditional journal articles and even supplemental materials lack the structure and permanence needed for detailed, evolving imaging protocols.
- Online Repositories Offer a Solution: Platforms like protocols.io provide a flexible, accessible, and citable space for researchers to share and update detailed protocols, overcoming the limitations of print-based publications.
- Standardization Enhances Usability: Creating modality-specific templates ensures a consistent structure, making it easier for researchers to write and find protocols across different imaging techniques.
Key Results
The paper presents the development and proposed implementation of standardized protocol templates for pre-clinical PET, CT, and MRI. These templates are now available on protocols.io. The authors anticipate that these PIPs will:
- Improve the reproducibility of pre-clinical imaging studies.
- Facilitate the sharing of detailed methods and instrument validation.
- Enable researchers to more easily adapt and build upon existing protocols.
- Provide a sustainable historical record of imaging methodologies.
- Potentially accelerate clinical translation by providing robust pre-clinical data.
Limitations and Future Work
- Concentration Risk: The reliance on protocols.io presents a potential risk if the platform changes its terms or becomes unavailable. GitHub is suggested as a backup.
- Philosophical Risk: The paper acknowledges the possibility that some pre-clinical imaging experiments may be inherently irreproducible due to factors like vendor variability, which PIPs might help highlight.
- Adoption and Utility: The long-term success of the PIPs will depend on their adoption and perceived utility by the research community.
- Future Directions:
- Incorporating feedback from early adopters to refine templates.
- Developing templates for other modalities like MRS, SPECT, and optical imaging.
- Facilitating language localization of templates and protocols through international collaborations.
- Creating a public database of PIPs with a user-friendly interface.
- Harmonizing PIP data with pre-clinical DICOM formats for automated metadata capture.
Summary updated