Abstract
As artificial intelligence (AI) becomes increasingly prevalent within interventional radiology (IR) research and clinical practice, steps must be taken to ensure the robustness of novel technological systems presented in peer-reviewed journals. This report introduces comprehensive standards and an evaluation checklist (iCARE) that covers the application of modern AI methods in IR-specific contexts. The iCARE checklist encompasses the full "code-to-clinic" pipeline of AI development, including dataset curation, pre-training, task-specific training, explainability, privacy protection, bias mitigation, reproducibility, and model deployment. The iCARE checklist aims to support the development of safe, generalizable technologies for enhancing IR workflows, the delivery of care, and patient outcomes.
Introduction
Interventional Radiology (IR) is a high-stakes, data-rich field where clinical decisions are guided by high-dimensional, spatiotemporal, and multimodal data. The integration of real-time fluoroscopy with pre-operative CT or MRI scans offers immense potential for AI optimization. However, the field currently faces a high rate of project failure due to limited data availability, algorithmic bias, and poor generalization to real-world clinical settings.
Existing reporting guidelines for medical AI, such as CLAIM, are often too generalized for the specific needs of IR. They frequently overlook the nuances of modern AI development, including the use of foundation models, complex data fusion, and the practicalities of procedural deployment. The iCARE (Interventional Radiology Reporting Standards and Checklist for Artificial Intelligence Research Evaluation) framework was developed to address these gaps, ensuring that AI tools are not only technically sound but also safe, ethical, and clinically viable.
The iCARE Framework: A "Code-to-Clinic" Approach
The iCARE checklist provides a structured evaluation across seven key domains, following the entire lifecycle of an AI model from initial data ingestion to clinical deployment:
- Pre-training Methodologies: Requires explicit reporting on the use of foundation models (e.g., LLMs or large-scale CNNs) and the nature of the data used to build them.
- Task-specific Training & Validation: Focuses on dataset curation and ensures that test sets are truly independent to prevent data leakage.
- Computational Methods: Investigates model architecture and the specific methods used to fuse multimodal data (e.g., early vs. late fusion).
- Explainability (Interpretation): Encourages tools like SHAP, LIME, or Grad-CAM and mandates the involvement of IR clinicians in validating these interpretations.
- Bias, Privacy, and Consent: Identifies 16 specific types of bias—from socioeconomic status to vendor-specific equipment bias—and requires detailed mitigation strategies.
- Code and Data Access: Introduces an "AI-readiness" rating for shared data, ranging from raw/unstructured to "High AI-readiness" (ready for immediate model input).
- Clinical Deployment: Categorizes the "level of disruption" the model brings to clinical workflows, ranging from "No disruption" to "Transformative disruption."
Standardizing Data Reporting
A significant portion of the iCARE framework is dedicated to how researchers report their datasets. It categorizes data into four distinct types to improve transparency:
- Proprietary: No researcher access to the raw source data.
- Open-source: Publicly available data (e.g., Common Crawl).
- Mined: Collected from public sources and preprocessed by the research team.
- Curated: Collected by researchers but not publicly accessible.
By emphasizing "AI-readiness," the checklist pushes researchers to document their preprocessing and filtering steps, which is essential for improving the reproducibility of AI research in the field.
Key Insights and Clinical Integration
The iCARE checklist introduces several paradigm shifts in AI evaluation for IR:
- The Shift to Pre-training: It recognizes that modern AI often starts with a pre-trained "foundation" model, forcing researchers to justify and document this often-opaque part of the pipeline.
- Defining "Disruption": The "Level of Disruption" scale forces developers to consider the human cost of implementation—whether a tool requires minimal training or a complete overhaul of clinical operations.
- Clinician-in-the-Loop: The framework explicitly asks if IR professionals were involved in data annotation and the assessment of model explainability, bridging the gap between mathematical accuracy and clinical meaningfulness.
- Standardizing Bias: By listing 16 specific potential sources of bias, iCARE moves beyond generic instructions, forcing researchers to look at IR-specific issues like the evolution of technology over time.
Conclusion
The primary result of this work is the iCARE Checklist itself, which has been formally endorsed by the Cardiovascular and Interventional Radiological Society of Europe (CIRSE) and the Society of Interventional Radiology (SIR). While the checklist relies on self-reporting and diligent adoption by journals and peer reviewers, it serves as a vital foundation for raising the quality and safety of AI research in Interventional Radiology. As AI continues to evolve, particularly with generative applications, the iCARE framework will serve as a living document to track and guide the progression of the field.
Summary updated