Radiologists reading lung CT tend to call an uncertain nodule "probably cancer" so they do not miss a malignancy. A large re-labeling study of the public LIDC-IDRI lung nodule database showed how costly this habit is. More than half of the nodules that radiologists scored as "likely malignant" turned out to be benign once checked against biopsy-proven references. A model trained on the radiologists' scores inherited the bias, with sensitivity 0.97 and specificity 0.23 (Re-thinking and Re-labeling LIDC-IDRI). That paper fixed the labels after the fact with an algorithm. It did nothing about the readers who produce such labels every day.
We propose to retrain the readers. The training splits into two parts, as in a flipped classroom. In self-paced online practice, trainees score nodules and then immediately see the biopsy result. Alongside it they see a radiomic contrast: quantitative texture, shape, and intensity measurements computed from the CT, plus the most similar biopsy-proven nodules by those measurements. A short live session follows, built around the specific nodule types where the practice data show each group over-calling. The question is whether this training improves readers' ability to tell benign from malignant indeterminate nodules on held-out biopsy-proven cases. Moving their threshold toward "benign" is not enough. A simple statistical recalibration could do that without any teaching at all.
Core Claim
Asynchronous practice with pathology feedback and radiomic contrasts improves reader discrimination on indeterminate nodules, measured as area under the ROC curve (AUC) on biopsy-proven cases. Pathology feedback alone, or a statistical shift of the reader's threshold, does not produce this gain. Choosing live-session content from the practice error data adds a further gain over a fixed live session. The new element is twofold. Radiomic features are used as a teaching signal for human readers rather than as model inputs. And the outcome is objective diagnostic accuracy against pathology, not the satisfaction surveys and course exams that dominate the education evidence.
What Already Exists and Where This Differs
The education literature motivates the design but leaves the key outcome unmeasured. A meta-analysis of 19 radiology studies found that flipped classrooms beat lectures on theory (standardized mean difference 1.12) and practical skills (SMD 2.59) (Ge et al.). Every intervention it covered taught qualitative visual reading, though. None measured specificity or over-calling.
A problem-based learning meta-analysis found much larger effects in hands-on laboratory formats than in lecture halls (SMD 2.01 vs 0.72) (Zhang et al. 2015). It also reported weak study quality, with a mean Jadad score of 3.6 out of 8. That result shapes our design in two ways: the core of the training is hands-on case practice, and the study is randomized against pathology rather than survey-based.
The closest work bringing blended learning to medical AI is a postgraduate curriculum designed with Kern's six steps (Cai et al.). It was validated only by expert opinion and teaches AI literacy broadly. We instead test a narrow, measurable skill.
Two further strands shape the delivery. Online-merge-offline designs, in which data from online activity steer what happens in class (Yang; Zhao et al.), supply the mechanism for the targeted live session. However, they were evaluated only by satisfaction. A survey found radiological anatomy acceptable online while gross anatomy was not (Sarkar et al.). That supports putting image practice online. The same survey found that 79% of students used smartphones and over a third reported eye pain or headaches, which limits how heavy the practice interface can be.
On the radiomics side, the feature families we use come from established practice (Aerts et al.): first-order intensity statistics, shape descriptors such as sphericity, co-occurrence texture such as gray-level co-occurrence matrix (GLCM) entropy, and wavelet-filtered texture. That same work warns that these features shift with slice thickness and reconstruction kernel, so we show trainees only features that are stable across those settings. A PET/CT deep radiomics model reached an AUC of 0.81–0.86 for EGFR mutation status (Nature Communications 2020). It shows that sub-visual signal exists, but it never asks whether humans can learn to use it.
Research Program
The first component is a case bank. Pathology-proven nodules from SCH-LND (180 solid nodules, 90 benign and 90 malignant, used as anchors in Zhang et al. 2022) are split by patient. Roughly 100 go to practice and 80 (40 benign, 40 malignant) form a sealed test set. If SCH-LND cannot be obtained, an institutional biopsy-proven set built to the same specification replaces it.
The practice pool is extended with LIDC-IDRI nodules. Only nodules whose re-labeled and original classifications agree are used, and they are marked "reference label, not pathology" so trainees are not drilled on noisy labels. Indeterminate nodules, those radiologists score 3 or 4, are deliberately over-sampled because that is where the bias lives.
The second component is the radiomic contrast panel. For each practice nodule we precompute a small set of reproducible features. Reproducibility is screened across resampled slice thicknesses and reconstruction kernels where available. The panel shows where the nodule sits relative to the benign and malignant distributions, together with its five nearest pathology-proven neighbors by those features. Everything renders as 2D slices and simple plots, so it works on a phone. No volumetric rendering happens on the trainee's device, which responds to the device and fatigue constraints Sarkar et al. report.
The third component is the live session. A 60-minute session per cohort goes through the indeterminate nodule types with the highest group over-calling rate in the practice logs. Typical examples are smooth nodules with low texture entropy that readers still score as 4. The comparison condition is the same 60 minutes covering a fixed, instructor-chosen case list.
Baselines
The naive reference point is untrained readers. Each trainee scores the sealed test set on a 1–5 scale before training. Their AUC and specificity at sensitivity matched to the trained arms form the do-nothing floor. No reported result was found on Lacuna for human reader AUC or specificity on SCH-LND, so this floor is measured, not assumed.
The second baseline is post-hoc recalibration of untrained readers. A logistic recalibration of each reader's pre-test scores is fit on the practice pool and applied to the test set. This moves thresholds without teaching anything. Training that only matches recalibration has taught readers nothing beyond "say benign more often." Beating this baseline means a statistically significant gain in AUC, since recalibration leaves AUC unchanged, and higher specificity at equal sensitivity.
The third baseline is the strongest existing method on this exact problem: the Siamese 3D ResNet-18 trained on re-labeled LIDC-IDRI from Zhang et al.
- It reports sensitivity 0.67 and specificity 0.66, against 0.97 and 0.23 for the model trained on the original labels. We retrain it from the released recipe on the same practice data and score it on the same sealed test set. That gives one shared operating point and AUC for humans, model, and model-assisted humans.
The ablation that isolates the key ingredient is a pathology-feedback-only arm. It gets identical practice cases, number, timing, and live session, but no radiomic panel. If this arm matches the full arm, the radiomic content is decoration. The live-session ablation contrasts targeted and fixed sessions with everything else held constant. All arms get equal training time, about four hours of practice plus one hour live, and the same case order rules.
Why not just…?
Why not just give readers the re-labeled AI model as a decision aid? This is a real threat. The model's reported 0.66 specificity at 0.67 sensitivity may beat trained humans. We therefore include an AI-assisted reading condition at test time for every arm. Training is worth doing only if trained readers paired with the model beat untrained readers paired with the model. That could happen if training helps readers catch the model's confident errors. If it does not, we report that the aid alone suffices.
Why not just show the biopsy answer after each case, without radiomics? Immediate outcome feedback alone may correct over-calling, and it is the cheapest intervention. This is exactly the pathology-feedback-only arm. The radiomic panel earns its place only if the full arm beats it on AUC, not merely on specificity.
Why not just recalibrate readers' scores statistically? Because recalibration shifts the cutoff and cannot improve ranking. The Zhang et al. 2022 finding that half of score-4 nodules are benign could reflect a pure threshold shift. If so, recalibration should match training, and the recalibration baseline will reveal it. Our claim survives only if training raises AUC.
Proposed Experiments and Decisive Checks
We will run a randomized four-arm trial with radiology residents and medical students from at least two sites, roughly 30 per arm. The arms are pathology feedback only, feedback plus radiomic panel with a fixed live session, feedback plus radiomic panel with a targeted live session, and a lecture-only control matched for time. Each participant reads the sealed 80-nodule test set before training, immediately after, and four weeks later, without and then with the AI aid.
The primary outcome is the change in reader AUC on indeterminate test nodules. Secondary outcomes are specificity at matched sensitivity, calibration, and retention at four weeks. Readers and cases are analyzed with a multi-reader multi-case model so that both kinds of variance are counted.
The first decisive check is whether the full arm's AUC gain over the feedback-only arm remains after both arms are recalibrated. If it vanishes, radiomic teaching only moved thresholds and the core claim fails. The second is whether trained readers' scores become more correlated with the stable radiomic features. A gain that comes without that correlation would point to a different mechanism, such as better attention to margins. The third is whether trained readers plus the AI aid beat the model alone on the same test set. Completion rates of the online practice are logged throughout. That tells us whether the drop-off seen when practice was purely self-paced (Manger et al.) also appears when a live session depends on the practice data.
References
- Re-thinking and Re-labeling LIDC-IDRI for Robust Pulmonary Cancer Prediction
- Effectiveness of flipped classroom vs traditional lectures in radiology education
- The Effectiveness of the Problem-Based Learning Teaching Model for Use in Introductory Chinese Undergraduate Medical Courses: A Systematic Review and Meta-Analysis
- Developing a Postgraduate Program for AI in Medicine with Kern's Six-Step Curriculum Development Approach in Singapore
- Study on the Application of Digitized Resources in English Language Teaching under a Blended Teaching Model
- Research and Practice of “M + LIF” Information-Based Teaching Application Mode Based on Teacher-Student Dual Subject
- Perception and attitude toward online versus traditional anatomy teaching: An internet-based cross-sectional study among Indian medical students during COVID pandemic
- Correction: Corrigendum: Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach
- Non-invasive decision support for NSCLC treatment using PET/CT radiomics
- Is Online Teaching Dead After COVID-19? Student Preferences for Programming Courses