Measures to increase value of preclinical research - an inexpensive and easy-to-implement approach to a QMS for an academic research lab

Michael Hewera, Ann-Christin Nickel, Nina Knipprath, Sajjad Muhammad, Xiaolong Fan, Hans-Jakob Steiger, Daniel Hänggi, Ulf Dietrich KahlertF1000Research 2020 · DOI

TL;DR

Academic research labs often struggle with reproducibility and translating findings due to a lack of standardized quality management. This paper presents a practical, low-cost Quality Management System (QMS) for academic labs, integrating readily available tools like an electronic lab notebook, SOPs, and a biobank to improve data quality, transparency, and efficiency.

What Problem Does This Paper Address?

The paper tackles the "reproducibility crisis" in science, where research findings are difficult to replicate, hindering progress and wasting resources. This is particularly challenging in academic research labs, which often operate with limited budgets and personnel. The authors highlight that while many solutions exist for industry, few are tailored to the specific constraints and needs of academic settings. They aim to provide a roadmap for implementing a QMS that is both inexpensive and easy to set up, thereby increasing the value and output of preclinical research.

Methods

The authors propose a QMS built around several key components:

  • eLabFTW (Electronic Lab Notebook): This open-source software serves as the central hub for all QMS data. It allows for detailed recording of experiments, storage of SOPs, lab meeting protocols, and automated data. Its key features include a non-delete policy, automatic logging of changes, and timestamping of entries, ensuring data integrity.
  • Standard Operating Procedures (SOPs): The implementation of standardized protocols for experiments aims to reduce individual differences in techniques and improve intra-lab reproducibility. SOPs are stored within eLabFTW and linked to specific processes.
  • Managed Bio Bank: A centralized system for storing bio samples (e.g., cell pellets, DNA) is established to ensure long-term preservation and facilitate future replications or sharing of lab tools. This includes a deep freezer and a liquid nitrogen tank, with a separate SQL database for tracking samples.
  • Lab Meetings: Regular meetings serve as a feedback mechanism for the QMS, allowing for evaluation of rules, discussion of improvements, and ensuring stakeholder engagement.
  • Automation: The paper discusses the implementation of automated systems, such as a robotic liquid handler, to create uniform data, eliminate individual differences, and enable direct data reporting into the electronic lab documentation system.

Datasets

The paper references STR (Short Tandem Repeat) analysis results for cell line authentication as a key dataset used for quality assessment. Specific data files related to STR analysis of BTSC 349, BTSC 268, and their mix are cited as being available on Zenodo. No new benchmarks were introduced; rather, existing methods and data types were integrated into the QMS framework.

Key Insights and Ideas

  • Democratizing Quality: The QMS is designed to be accessible and affordable for academic labs, demonstrating that robust quality management doesn't require significant financial investment.
  • eLabFTW as a Central Hub: The choice of eLabFTW as an open-source, flexible platform is crucial for integrating various QMS components, making it a practical solution for labs.
  • Beyond Data Integrity: The QMS is not just about data accuracy; it also fosters improved communication, teamwork, and a sense of shared responsibility within the lab.
  • Proactive Quality Control: Implementing routine quality checks like STR analysis and mycoplasma testing for cell lines is presented as a cost-effective way to prevent the generation of low-value data.
  • Automation for Reproducibility: The integration of automation, even simple forms like pipetting pumps, can significantly enhance data uniformity and reduce errors.

Key Results

While the paper focuses on the implementation and benefits of a QMS, it reports positive outcomes such as:

  • Reduced individual differences in work techniques.
  • Improvements in sample and data storage.
  • Increased stakeholder engagement.
  • Promotion of new scientific discoveries (e.g., identifying a repurposed drug for brain tumor cells through automation-assisted screening).
  • Enhanced confidence in scientific results due to improved transparency and reproducibility.
  • Standardization of 91 unique procedures and recipes.

Limitations and Future Work

  • Language and Clarity: Reviewers noted several issues with English usage and sentence structure, suggesting the manuscript would benefit from a thorough language revision.
  • Data for Claims: Some claims, such as increased team motivation and consistency of experimental results, were not fully substantiated with quantitative data. The authors were encouraged to provide more evidence or clearly mark these as speculative.
  • Specificity of Methods: The detailed description of STR analysis methods was considered excessive for the main text and potentially better suited for supplementary material.
  • Cost and Implementation Details: More information on the financial costs and human resource requirements for implementing each QMS block was requested to fully support the claim of a "resource-restricted" QMS.
  • Inequality and Discrimination: The link between QMS implementation and the reduction of inequality/discrimination was not clearly explained and requires further clarification.
  • Future Work: The authors suggest that further projects are needed to compare manual pipetting data with robot-derived data to conclusively evaluate the impact of automation on data quality. They also acknowledge the need for more detailed information on the implementation costs and challenges of their QMS.

Summary updated

BibTeX
@article{hewera2020measures,
  author = {Michael Hewera and Ann-Christin Nickel and Nina Knipprath and Sajjad Muhammad and Xiaolong Fan and Hans-Jakob Steiger and Daniel Hänggi and Ulf Dietrich Kahlert},
  title = {{Measures to increase value of preclinical research - an inexpensive and easy-to-implement approach to a QMS for an academic research lab}},
  journal = {F1000Research},
  year = {2020},
  doi = {10.12688/f1000research.24494.1},
}
Figure