Open Development and Clinical Validation of Multiple 3D-Printed Nasopharyngeal Collection Swabs: Rapid Resolution of a Critical COVID-19 Testing Bottleneck

Cody Callahan, Rose Lee, Katelyn E. Zulauf, Lauren Tamburello, Kenny Smith, Joe Previtera, Annie Cheng, Alexander L. Green, Ahmed Abdul Azim, Amanda Yano, Nancy Doraiswami, James E. Kirby, Ramy ArnaoutJournal of Clinical Microbiology 2020 · DOI

TL;DR

During the COVID-19 pandemic, a critical shortage of nasopharyngeal swabs threatened diagnostic testing capacity. This paper details an innovative, open-source, collaborative effort that rapidly designed, pre-clinically evaluated, and clinically validated multiple 3D-printed swab prototypes, successfully addressing this bottleneck within weeks.

What Problem Does This Paper Address?

The COVID-19 pandemic created an unprecedented global demand for nasopharyngeal (NP) swabs, essential for collecting optimal specimens for highly sensitive SARS-CoV-2 reverse transcriptase PCR (RT-PCR) testing. This surge in demand, coupled with manufacturing disruptions, led to severe international shortages, creating a critical bottleneck that hampered clinical laboratories' ability to perform widespread testing. Existing commercial swabs, while sometimes repurposed, were often suboptimal due to differences in size, flexibility, and potential PCR inhibition. The paper addresses the urgent need for a rapid, scalable solution to this swab shortage.

Methods

The authors employed a multi-phase, open-development approach:

  • Phase 1: Preclinical Evaluation: A rigorous evaluation of 160 swab designs and 48 materials from 24 entities was conducted. This involved assessing mechanical properties (flexibility, durability, break point), surface smoothness, adhesiveness, odor, collection sufficiency (using Gram staining of cheek swabs as a surrogate), and crucially, PCR compatibility (testing for inhibition of SARS-CoV-2 RT-PCR).
  • Phase 2: Production Considerations: Manufacturers were assessed for their ability to produce at least 10,000 swabs per day and their short-term strategies for packaging and sterilization (e.g., autoclaving). Supply chain considerations were also factored in.
  • Phase 3: Clinical Validation (Field Testing): An IRB-approved clinical trial was designed to compare prototype swabs against a standard control swab. Outpatients presenting with COVID-19 symptoms were swabbed with both a control and a prototype. RT-PCR results were compared for concordance, and cycle threshold (C<sub>T</sub>) values were analyzed to assess noninferiority.
  • Open Development Platform: A key methodological innovation was the use of a public GitHub repository to share design specifications, test results, and feedback. This facilitated rapid iteration and collaboration among healthcare workers, manufacturers, and scientists.

Datasets

The paper does not introduce a new benchmark dataset in the traditional machine learning sense. Instead, it leverages:

  • Preclinical Test Data: Results from the evaluation of 160 swab designs and 48 materials, documented in a public Excel spreadsheet.
  • Clinical Trial Data: Swab samples collected from 276 outpatient volunteers. These samples were tested using the Abbott RealTime SARS-CoV-2 assay.
  • Quality Control Data: Data from 88 cases where patients underwent duplicate swabbing within 24 hours for quality control purposes, used to compare variability between replicate control swabs and control vs. prototype swabs.

Key Insights and Ideas

  • The Power of Open Collaboration: The study powerfully demonstrates how an open, decentralized, and collaborative approach can rapidly solve critical real-world problems, outperforming closed or proprietary models, especially under time pressure.
  • 3D Printing as a Rapid Prototyping Solution: 3D printing proved to be an agile technology, allowing for quick design iteration and adaptation, which was crucial during the pandemic's fast-evolving landscape.
  • Non-Flocked Swabs Can Be Effective: Notably, none of the validated prototypes were flocked. This challenges the assumption that flocked swabs are inherently superior and highlights that alternative designs can achieve comparable performance.
  • Speed of Development is Achievable: The remarkable timeline of 22 days from problem identification to validation of the first prototype underscores the potential for rapid translational research when resources and collaboration are aligned.
  • Clinical Validation is Paramount: While preclinical testing is essential, the paper emphasizes the necessity of rigorous clinical validation to ensure that novel devices perform effectively in real-world patient care.

Key Results

  • Successful Validation: Four 3D-printed prototypes successfully passed preclinical and clinical validation.
  • High Concordance: All four prototypes demonstrated excellent categorical concordance with the control swab, with Cohen's kappa (κ) values ranging from 0.85 to 0.89.
  • Noninferior Performance: C<sub>T</sub> values for the prototypes were not significantly different from the control swab (Mann-Whitney U test, P > 0.05), indicating noninferior performance in detecting SARS-CoV-2.
  • Staff Preferences: While study staff generally preferred the control swab overall, prototype 4 was the most preferred among the prototypes. Narrative feedback indicated prototype 4 was comparable to the control.
  • Rapid Timeline: The entire process from identifying the problem to validating the first prototype took just 22 days.
  • Scalable Production: Manufacturers indicated the ability to produce at least 10,000 swabs per day.

Limitations and Future Work

  • Sample Size for Preference: While the clinical trial involved 276 participants, the staff preference survey involved smaller numbers for each prototype (n=18 for prototype 1, n=14 for prototype 2, n=8 for prototype 3, and n=7 for prototype 4).
  • Potential for Bias in Swabbing: The paper acknowledges potential sources of variance, such as differences in secretions or viral burden between nares, and the possibility that the first swab might leave less material for the second.
  • "Perfect" vs. "Good Enough": The authors note that in a crisis, the pursuit of perfection can hinder progress, and the focus should be on achieving "good enough" solutions that can be deployed quickly.
  • Future Work: The paper suggests that complementary manufacturing techniques may play a role in a more mature market. The ongoing clinical trial will continue to evaluate other prototypes. The experience gained is intended to inform future rapid development and deployment of medical technologies during pandemics.

Summary updated

Abstract

The pandemic caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused a severe international shortage of the nasopharyngeal swabs that are required for collection of optimal specimens, creating a critical bottleneck blocking clinical laboratories' ability to perform high-sensitivity virological testing for SARS-CoV-2. To address this crisis, we designed and executed an innovative, cooperative, rapid-response translational-research program that brought together health care workers, manufacturers, and scientists to emergently develop and clinically validate new swabs for immediate mass production by 3D printing. We performed a multistep preclinical evaluation of 160 swab designs and 48 materials from 24 companies, laboratories, and individuals, and we shared results and other feedback via a public data repository (http://github.com/rarnaout/Covidswab/). We validated four prototypes through an institutional review board (IRB)-approved clinical trial that involved 276 outpatient volunteers who presented to our hospital's drive-through testing center with symptoms suspicious for COVID-19. Each participant was swabbed with a reference swab (the control) and a prototype, and SARS-CoV-2 reverse transcriptase PCR (RT-PCR) results were compared. All prototypes displayed excellent concordance with the control (κ = 0.85 to 0.89). Cycle threshold (<i>C<sub>T</sub></i> ) values were not significantly different between each prototype and the control, supporting the new swabs' noninferiority (Mann-Whitney U [MWU] test, <i>P</i> > 0.05). Study staff preferred one of the prototypes over the others and preferred the control swab overall. The total time elapsed between identification of the problem and validation of the first prototype was 22 days. Contact information for ordering can be found at http://printedswabs.org Our experience holds lessons for the rapid development, validation, and deployment of new technology for this pandemic and beyond.
BibTeX
@article{callahan2020open,
  author = {Cody Callahan and Rose Lee and Katelyn E. Zulauf and Lauren Tamburello and Kenny Smith and Joe Previtera and Annie Cheng and Alexander L. Green and Ahmed Abdul Azim and Amanda Yano and Nancy Doraiswami and James E. Kirby and Ramy Arnaout},
  title = {{Open Development and Clinical Validation of Multiple 3D-Printed Nasopharyngeal Collection Swabs: Rapid Resolution of a Critical COVID-19 Testing Bottleneck}},
  journal = {Journal of Clinical Microbiology},
  year = {2020},
  doi = {10.1128/jcm.00876-20},
}

Loading related content...

Figure