Recommendations for a step‐wise comparative approach to the evaluation of new screening tests for colorectal cancer

Graeme P. Young, Carlo Senore, Jack S. Mandel, James E. Allison, Wendy Atkin, Robert Benamouzig, Patrick M. Bossuyt, Mahinda De Silva, Lydia Guittet, Stephen P Halloran, Ulrike Haug, Geir Hoff, Steven H. Itzkowitz, Mārcis Leja, Bernard Levin, Gerrit A. Meijer, Colm O’Morain, Susan Parry, Linda Rabeneck, Paul Rozen, Hiroshi Saito, Robert E. Schoen, Helen E. Seaman, R. Steele, Joseph J.�y. Sung, Sidney J. WinawerCancer 2016 · DOI

TL;DR

This paper proposes a structured, four-phase approach for evaluating new colorectal cancer (CRC) screening tests. It emphasizes a comparative methodology, moving from initial accuracy assessments to large-scale population studies, ensuring that new tests are rigorously compared against established ones before widespread adoption. This framework aims to provide a rational and efficient pathway for introducing effective new screening technologies while minimizing risks and costs.

What Problem Does This Paper Address?

The development of new screening tests for colorectal cancer (CRC) is ongoing, but their adoption into screening programs often outpaces the evidence needed to justify their use. A significant challenge lies in efficiently and rigorously evaluating these new tests, especially when compared to existing, proven methods. Simply assessing clinical accuracy (like sensitivity and specificity) is insufficient; a comprehensive evaluation must also consider safety, public acceptability, cost-effectiveness, and programmatic impact. Without a standardized, step-wise evaluation framework, promising new technologies might be adopted prematurely or face unnecessary delays, impacting public health outcomes.

Methods

The authors propose a four-phase, step-wise comparative evaluation framework:

  • Phase 1: Retrospective Estimation of Ability to Discriminate: This initial phase focuses on estimating the test's ability to distinguish between cancer cases and controls without neoplasia. It uses retrospective data and "paired testing" (where individuals undergo both the new and comparator tests) to assess accuracy measures like true-positive and false-positive rates.
  • Phase 2: Detection of Neoplasia Across the Oncogenic Continuum: This phase involves prospective clinical studies to evaluate test performance across various stages of CRC development, from early neoplasia to advanced adenomas. It aims to fine-tune the test format and endpoint and assess performance in asymptomatic individuals.
  • Phase 3: Initial Screening Evaluation (Single Round): This phase moves to a screening context, typically involving randomized cohorts performing either the new or comparator test. It assesses participation rates, prevalence of neoplasia, workload, and test acceptability in a "real-world" screening setting.
  • Phase 4: Screening Program Evaluation (Multiple Rounds): The final phase involves evaluating the new test's performance over multiple rounds in a large, unselected population. This phase focuses on long-term outcomes like participation, detection rates, interval cancers, adverse events, and cost-effectiveness, aiming to confirm the test's benefit in an organized screening program.

The framework emphasizes a comparative approach throughout, ensuring new tests are benchmarked against proven ones.

Datasets

Not discussed in the paper. The paper focuses on the methodology for evaluating tests rather than presenting results from specific datasets.

Key Insights and Ideas

  • Phased Evaluation is Crucial: A step-wise approach, increasing in complexity and cost only if earlier phases show promise, is more efficient and rational than attempting full evaluation at once.
  • Comparison is Paramount: New tests must be directly compared to proven comparator tests (like gFOBT or FIT) at each stage to understand their relative benefit and potential harms.
  • Mortality is Not Always the Primary Endpoint: While the ultimate goal is mortality reduction, intermediate endpoints like test accuracy, participation, and detection rates can predict benefit and are more feasible to measure in earlier phases.
  • Programmatic Factors Matter: Beyond technical accuracy, factors like participation, acceptability, and feasibility are critical for a screening test's success in a population setting.
  • Ethical Considerations: It's unethical to proceed with population-level studies without preliminary evidence of acceptable accuracy compared to a proven test.

Key Results

The paper does not present empirical results from a specific study. Instead, it outlines a framework and provides guidance on how to conduct such evaluations. The "key results" are the principles and recommendations for the phased evaluation process itself.

Limitations and Future Work

  • Cost: The later phases (especially Phase 3 and 4) are acknowledged to be very costly, potentially running into millions of dollars.
  • Generalizability: The authors note that without careful characterization of study groups and detected lesions, transferability of findings from one setting to another may not be possible.
  • New Biomarkers: The paper briefly touches on the added complexity of evaluating new biomarkers, which may require fine-tuning of test endpoints in early phases.
  • Future Work: The paper implicitly calls for the application of this framework to evaluate emerging CRC screening technologies. It also suggests that further research is needed to refine cost-effectiveness analyses and to understand the long-term impact of new tests over multiple screening rounds. The authors also mention that ongoing population screening trials evaluating colonoscopy will provide valuable data for its use as a comparator.

Summary updated

Abstract

New screening tests can be evaluated efficiently by this stepwise comparative approach.
BibTeX
@inproceedings{young2016recommendations,
  author = {Graeme P. Young and Carlo Senore and Jack S. Mandel and James E. Allison and Wendy Atkin and Robert Benamouzig and Patrick M. Bossuyt and Mahinda De Silva and Lydia Guittet and Stephen P Halloran and Ulrike Haug and Geir Hoff and Steven H. Itzkowitz and Mārcis Leja and Bernard Levin and Gerrit A. Meijer and Colm O’Morain and Susan Parry and Linda Rabeneck and Paul Rozen and Hiroshi Saito and Robert E. Schoen and Helen E. Seaman and R. Steele and Joseph J.�Y. Sung and Sidney J. Winawer},
  title = {{Recommendations for a step‐wise comparative approach to the evaluation of new screening tests for colorectal cancer}},
  booktitle = {Cancer},
  year = {2016},
  doi = {10.1002/cncr.29865},
}
Figure