Abstract
CCT-Net addresses the "annotation bottleneck" in medical AI by transferring knowledge from a well-studied disease with pixel-level labels (source) to multiple related diseases that only have image-level labels (target). By learning a category-invariant way to refine coarse disease localizations into fine-grained masks, the model significantly improves both the localization and classification of new diseases without needing additional expensive manual segmentations.
Introduction
In medical imaging, expert pixel-level annotation is time-consuming and expensive. Consequently, while "rich" datasets exist for common conditions like Diabetic Retinopathy, clinical reality requires diagnosing multiple diseases simultaneously, many of which may only have simple image-level tags. Existing transfer learning methods often require source and target diseases to be identical or necessitate extensive fine-tuning. CCT-Net tackles the single-to-multiple disease diagnosis challenge: leveraging the "spatial intelligence" learned from one well-annotated disease to identify and localize several related diseases that lack fine-grained maps.
Methodology
The CCT-Net framework consists of three primary components designed to bridge the gap between source and target domains while refining disease localization.
- Domain-Specific Encoder (DSE): To account for style shifts between different scanners or populations, the model utilizes domain-specific Batch Normalization (BN). This allows the network to learn shared medical features while maintaining domain-specific statistics.
- Confidence Weighted Pooling (CWP): Unlike standard Global Average Pooling, CWP is a trainable module that assigns weights to image regions based on disease probability. This results in sharper and more accurate coarse heatmaps (Class Activation Maps).
- Category-Invariant Feature Refinement (CIFR): This is the core innovation. CIFR blocks learn a general refinement task: producing a pixel-level mask given a feature map and a coarse heatmap. Because the process is conditioned on the heatmap rather than a specific class, it is category-invariant and can be applied to entirely different diseases in the target domain.
Refined features from this process are subsequently processed by a second encoder (DSE-2) to enhance final image-level classification accuracy.
Experimental Results
The model was evaluated on two distinct medical imaging tasks: Ocular Disease diagnosis (Fundus images) and Brain Tumor classification (MRI).
- Ocular Task: Using FGADR (pixel-level Diabetic Retinopathy) as the source and ODIR-5K (image-level for 8 diseases) as the target, CCT-Net achieved a Kappa score of 0.7508, outperforming standard baselines like DANN (0.6934) and MCD (0.7296).
- Brain Task: Using BraTS 2019 (pixel-level glioma) as the source and a multi-class BrainTumor dataset as the target, the model reached an average accuracy of over 95% across meningioma, pituitary tumors, and glioma.
Notably, the model generated accurate segmentations for diseases it never saw pixel-level masks for during training (e.g., meningiomas), demonstrating the effectiveness of the category-invariant refinement.
Discussion and Conclusion
The primary contribution of CCT-Net is treating segmentation refinement as a category-agnostic skill. By conditioning refinement on coarse heatmaps, the network learns a general geometric logic that transfers across diseases. This approach demonstrates that improved localization directly enhances classification robustness. Future work may explore cross-modality transfer and more advanced weakly-supervised localization methods to further improve the initial "seed" heatmaps.
Summary updated