Abstract
This study demonstrates the added value of a Rule Set Table (RST) in enhancing the predictive performance of oncology-specific machine learning (ML) models. By integrating clinician domain knowledge into the data preprocessing (DP) phase, the research serves as a proof of concept for a more inclusive, clinician-driven AI development process.
Introduction
Machine learning in medical imaging typically relies on purely mathematical data preprocessing, often leaving valuable clinician domain knowledge out of the loop. While automated systems excel at statistical optimization, they lack the biological context that human experts provide. This paper introduces the "Rule Set Table" (RST) interface, which allows doctors to inject explicit or preferential rules into the preprocessing pipeline. By guiding feature selection with clinical expertise, the researchers significantly boosted the balanced accuracy of predictive oncology models.
Methodology
The core contribution of this work is the Rule Set Table (RST). It is a human-readable interface where clinicians can define rules that influence preprocessing algorithms.
The RST supports four primary actions:
- Explicit rules (
exp-keep,exp-remove): These are hard constraints. The algorithm must keep or drop a specific feature or sample, regardless of its mathematical ranking. - Preferential rules (
pref-keep,pref-remove): These are soft constraints. The algorithm is nudged to follow the clinician's preference but can overrule it if doing so would severely harm the model's statistical performance.
Researchers tested the RST on two types of preprocessing pipelines: a standard Manual DP (using common algorithms like SMOTE and Sequential Forward Selection) and a complex, fully automated ML-Driven DP (MLDP).
Datasets and Experimental Setup
The study utilized three distinct oncology cohorts with radiomic features extracted from PET/CT and PET/MRI scans:
- Glioma (Single Center, n=69): Predicting 36-month survival.
- Prostate Cancer (Single Center, n=57): Predicting high-vs-low risk.
- DLBCL (Dual Center, n=85): Predicting 24-month progression across two hospitals.
All models were built using the XGBoost classifier and evaluated using 100-fold Monte Carlo cross-validation or independent validation sets.
Key Results and Insights
- Significant Gains in Manual Pipelines: Adding RST rules to standard manual preprocessing pipelines increased Balanced Accuracy (BACC) by up to 18%. For the Glioma cohort, BACC rose to 81% (up from 63% without rules). The Prostate and DLBCL cohorts saw similar improvements.
- Stability in Automated Pipelines: While the fully automated MLDP pipelines performed well on their own (79-81% BACC), the RST remained valuable for providing clinicians with control and interpretability over otherwise opaque "black-box" systems.
- A Tool for Hypothesis Testing: The RST allows clinicians to test medical hypotheses directly. For example, a doctor could force a model to prioritize specific radiomic features to see if they yield better predictive performance than standard mathematical selections.
- Sensitivity to Knowledge Quality: The effectiveness of this approach relies on the quality of clinical input. When models were given rules to remove features that were actually highly predictive, performance dropped, proving the algorithm genuinely incorporates human guidance.
Discussion and Future Work
This study proves that human-in-the-loop preprocessing can bridge the performance gap between basic manual setups and complex AutoML systems. However, a limitation is its retrospective nature; the rules used were based on features already known to be predictive. Future work will focus on prospective testing, where clinicians input genuine hypotheses without prior knowledge of the "correct" statistical answer. This approach marks a significant step toward making AI pipelines in nuclear medicine more inclusive and clinically relevant.
Summary updated