TL;DR
This paper provides a comprehensive visual review of the factors contributing to variability in RECIST 1.1 assessments for cancer treatment response. It highlights how inconsistencies in lesion selection, new lesion detection, and image interpretation can impact clinical trial outcomes and patient management, emphasizing the need for standardized training and adherence to guidelines.
What Problem Does This Paper Address?
In oncology, accurately assessing a patient's response to treatment is crucial for determining treatment efficacy and guiding clinical decisions. This assessment often relies on imaging criteria, with RECIST (Response Evaluation Criteria in Solid Tumors) being a widely adopted standard. However, even with standardized criteria like RECIST 1.1, significant variability exists between radiologists in interpreting these images. This inter-observer variability can lead to inconsistent assessments of disease progression or response, potentially affecting clinical trial endpoints and, ultimately, patient care. The paper aims to dissect the root causes of this variability, particularly within the context of blinded, double-read assessments common in clinical trials, and to propose ways to mitigate it.
Methods
The paper employs a systematic pictorial review approach. It breaks down the RECIST 1.1 criteria into its core components: target lesions (TLs), non-target lesions (NTLs), and new lesions (NLs). For each component, the authors illustrate common sources of variability using clinical imaging examples. They categorize these sources into several key areas:
- Baseline Lesion Selection: This includes issues like non-representativeness, choosing different target lesions, or failing to select the largest lesions.
- Measurement Variability: Discrepancies in how lesions are measured, including issues with lesion definition, measurement methods, and acquisition parameters.
- Detection of New Lesions: This covers false positives (mistaking artifacts or benign findings for new lesions) and false negatives (missing new lesions).
- Assessment of Non-Target Lesions: The qualitative nature of assessing NTLs, which can be subjective.
- Image Quality and Contextual Information: The impact of poor image quality, lack of contrast, or missing patient clinical history on interpretation.
- Cognitive Biases: Such as confirmation bias, where radiologists might interpret findings to align with previous assessments.
The review uses a "toolbox" approach, presenting practical actions radiologists can take to minimize variability.
Datasets
Not discussed in the paper. The paper focuses on illustrating variability using real-world clinical imaging examples from various patients, rather than a specific, curated dataset for training or evaluation.
Key Insights and Ideas
- Baseline Selection is Paramount: The initial choice of target lesions significantly influences subsequent assessments. If the baseline selection isn't representative of the overall disease burden, it can lead to downstream discrepancies.
- "New Lesions" are a Major Pitfall: The detection and characterization of new lesions are frequent sources of discordance, with false positives and false negatives contributing substantially to variability. This is particularly critical as the appearance of a new lesion often triggers a "progressive disease" classification, impacting treatment decisions.
- Subjectivity Creeps In: While RECIST aims for quantification, many aspects, especially the assessment of non-target lesions and the distinction between scars and residual disease, remain qualitative and thus prone to subjective interpretation.
- Context is King (But Often Missing): In blinded central reviews, radiologists are often deprived of crucial clinical information. This lack of context can lead to diagnostic uncertainty and variability, while conversely, having too much context can introduce bias.
- Cognitive Biases are Real: Radiologists, like all humans, are susceptible to cognitive biases such as "satisfaction of search" or "confirmation bias," which can subtly influence their interpretations over time.
- Small Differences Matter: Even millimetric differences in lesion measurements can lead to different response classifications (e.g., stable disease vs. progressive disease), especially when dealing with small residual lesions or patients with a low tumor burden.
Key Results
The paper doesn't present quantitative results in terms of accuracy metrics or statistical comparisons. Instead, its key findings are qualitative and illustrative:
- Categorization of Variability Sources: The paper systematically identifies and visually demonstrates the primary causes of RECIST 1.1 variability across target lesions, non-target lesions, and new lesions.
- Pictorial Evidence: Through numerous figures, the paper provides clear visual examples of how different interpretations of the same imaging findings can occur, leading to discordance.
- "Toolbox" for Radiologists: A practical table (Table 2) summarizes actionable steps radiologists can take to mitigate variability, covering lesion selection, new lesion detection, NTL assessment, and target lesion measurement.
- Emphasis on Training and Standardization: The overarching message is that adherence to RECIST guidelines and comprehensive training are essential for reducing errors and improving consistency.
Limitations and Future Work
- Focus on Pictorial Review: While effective for illustration, this method doesn't provide quantitative data on the prevalence or impact of each variability factor.
- Lack of Specific Dataset: The paper doesn't introduce a new benchmark dataset, which could be valuable for developing and testing automated or semi-automated tools to reduce variability.
- Future Work Suggestions: The authors implicitly suggest that further research could focus on developing more objective or automated methods for lesion detection and measurement. They also highlight the importance of understanding how different trial phases (e.g., Phase 2 vs. Phase 3) might require different strategies for minimizing variability, with Phase 2 trials potentially needing closer monitoring of lesion measurement and selection, while Phase 3 trials might prioritize new lesion assessment. The paper also points to the need for better integration of contextual information into blinded review processes where feasible.
Summary updated