Development and Multicenter Assessment of a Reference Panel for Clinical Shotgun Metagenomics for Pathogen Detection

Donglai Liu, Teng Xu, Haiwei Zhou, Qiwen Yang, Xi Mo, Dawei Shi, Jingwen Ai, Jingjia Zhang, Yue Tao, Wen Donghua, Yigang Tong, Lili Ren, Wen Zhang, Shumei Xie, Weijun Chen, Wanli Xing, Jinyin Zhao, Yilan Wu, Xianfa Meng, Chuan Ouyang, Zhi Jiang, Zhikun Liang, Hai-Qin Tan, Yuan Fang, Nan Qin, Yuanlin Guan, Wei Gai, Sihong Xu, Wenjuan Wu, Chuntao Zhang, Youchun WangResearch 2021 · DOI

Abstract

Abstract Background Clinical shotgun metagenomics for pathogen detection has been used in diagnosing infectious diseases. However, its technical assessments have been limited to reference standards by individual labs, single experimental workflow and laboratory. Results Here we reported the design and development of a set of reference reagents and reporting metrics dedicated for clinical metagenomics by the National Institutes for Food and Drug Control (NIFDC) of China, and a joint evaluation study including 17 independent laboratories in cross-workflow and cross-platform settings. Our results showed that the performance of metagenomic assays was significantly impacted by the factors of microbial types, host context, and read depth, thus highlighting the importance to take these factors into consideration when designing reference reagents and benchmarking assays. Through this study, we found false positives to be a common challenge across centers, and considerable site and library effects to limited the assay’s quantitative value. Our multicenter study also provided practical guidance of performance that laboratory-developed shotgun pathogen metagenomics tests should aim to detect microbes at 500 CFU/mL (or copies/mL) in clinically relevant host context (10^5 human cells/mL) within a 24h turn-around time, and with a read depth of 20M reads or lower. This collaboration work provided a unique resource comprising nearly 600 billion reads (>5Tb) for technical evaluation in clinical and regulatory settings. Conclusion We demonstrated that the performance of metagenomic assays was significantly impacted by the microbial type, the host context and read depth, which emphasizes the importance to consider these factors when designing reference reagents and benchmarking studies. Across sites, workflows and platforms, false positive reporting and considerable site/library effects were common challenges to the assay’s accuracy and quantifiability. Our study also suggested practical guidance of performance that laboratory-developed shotgun pathogen metagenomics tests should aim to detect microbes at 500 CFU/mL (or copies/mL) in clinically relevant host context (10^5 human cells/mL) within a 24h turn-around time, and with a read depth of 20M reads or lower. This collaboration work provided a unique resource comprising nearly 600 billion reads (>5Tb) for technical evaluation in clinical and regulatory settings.
BibTeX
@inproceedings{liu2021development,
  author = {Donglai Liu and Teng Xu and Haiwei Zhou and Qiwen Yang and Xi Mo and Dawei Shi and Jingwen Ai and Jingjia Zhang and Yue Tao and Wen Donghua and Yigang Tong and Lili Ren and Wen Zhang and Shumei Xie and Weijun Chen and Wanli Xing and Jinyin Zhao and Yilan Wu and Xianfa Meng and Chuan Ouyang and Zhi Jiang and Zhikun Liang and Hai-Qin Tan and Yuan Fang and Nan Qin and Yuanlin Guan and Wei Gai and Sihong Xu and Wenjuan Wu and Wenhong Zhang and Chuntao Zhang and Youchun Wang},
  title = {{Development and Multicenter Assessment of a Reference Panel for Clinical Shotgun Metagenomics for Pathogen Detection}},
  booktitle = {Research},
  year = {2021},
  doi = {10.21203/rs.3.rs-208796/v1},
}

Loading related content...

Figure