Introduction
Radiology and pathology have historically diverged in their approach to AI. While pathology leverages task-agnostic foundation models trained on 2D patches with attention mechanisms, radiology has largely relied on resource-intensive, end-to-end 3D networks tailored for specific tasks. These 3D models require massive annotated datasets and significant computational power.
To bridge this gap, researchers introduced CLEAR (Contrastive Learning-based Embeddings for Attention-based Radiology), a framework designed to learn general "abnormality" representations across the whole body using efficient 2D slices. This approach brings the efficiency of digital pathology workflows to the complex world of radiology imaging.
The CLEAR Framework and LECL
The core of CLEAR is the Lesion-Enhanced Contrastive Learning (LECL) strategy. Unlike standard contrastive learning, which simply matches different views of the same image, LECL is specifically "lesion-aware." During pre-training, it prioritizes crops centered on known abnormalities and employs a specialized loss function to distinguish abnormal slice embeddings from the rest of the training set.
The architecture moves beyond standard Vision Transformers (ViTs), exploring more efficient alternatives like Vision State Space Models (VMamba) and Gated CNNs (MambaOut). To process 3D volumes, CLEAR extracts 2D embeddings for each slice and aggregates them using Attention-based Multiple Instance Learning (ABMIL), which learns to weigh the clinical relevance of each slice.
Experimental Setup and Datasets
The model was pre-trained on DeepLesion, a dataset containing over 870,000 axial CT slices. Evaluation spanned three distinct tasks to test both internal and external generalization:
- Internal Validation: Lesion classification on DeepLesion across various body regions (Abdomen, Mediastinum, Pelvis, etc.).
- External Validation (RadChest): Lung disease detection (e.g., Emphysema, Fibrosis).
- External Validation (NSCLC-Radiomics): Patient cancer staging.
Key Insights and Results
The study yielded several critical findings that challenge the necessity of heavy 3D models:
- 2D Efficiency: High-quality 2D embeddings paired with smart attention mechanisms can effectively capture spatial context, often outperforming heavy 3D convolutions while being significantly more data- and compute-efficient.
- Architecture Matters: VMamba and MambaOut consistently outperformed standard ViTs, suggesting these architectures are better suited for the specific textures and structures found in medical imaging.
- Superior Performance: On lesion classification, the MambaOut-based LECL model achieved a +0.8% AUC improvement over BiomedCLIP. In specialized areas like soft tissue, the gain was a significant +5.9% AUC.
- Generalization: The model demonstrated strong performance on external datasets (RadChest and NSCLC) without requiring task-specific fine-tuning of the backbone.
Interpretability by Design
One of the primary advantages of the CLEAR framework is its inherent interpretability. By utilizing an attention-based aggregator, the model provides a clear signal regarding which slices were most influential in its final decision.
This transparency allows clinicians to verify the AI's reasoning, transforming the "black box" into a tool that aligns with clinical workflows. Doctors can see exactly which slices the AI used to make its decision, making the model's output more trustworthy and actionable.
Limitations and Future Directions
The researchers identified a "Goldilocks" effect with the lesion-weighting parameter ($\lambda$); excessive weighting can cause the model to ignore necessary healthy anatomical context. Future work will explore extending CLEAR to other modalities like MRI and PET, and integrating full 3D segmentation capabilities to further support clinical needs. Additionally, the authors suggest that further research into domain-specific architectures is key to unlocking the full potential of AI in radiology.
Summary updated