Introduction
Deep learning has revolutionized lung cancer diagnosis, yet a significant gap remains between model accuracy and clinical deployability. State-of-the-art architectures like Vision Transformers (ViTs) and deep Convolutional Neural Networks (CNNs) achieve high performance but are computationally prohibitive for real-time use in resource-constrained environments, such as mobile clinical devices. While Knowledge Distillation (KD) offers a path to compress these models by transferring knowledge from a large "teacher" to a smaller "student," traditional KD methods rely on fixed weights. This approach often fails in medical imaging, where noise and ambiguity require a more nuanced, adaptive learning strategy.
This paper introduces FuzzyDistillViT-MobileNet, a framework that utilizes fuzzy logic to dynamically adjust KD weights based on image uncertainty, enabling a lightweight MobileNet student to achieve performance comparable to heavy-duty architectures.
Methodology
Fuzzy Logic Dynamic Knowledge Distillation
The core innovation of this work is the fuzzy logic controller used during the distillation process. Traditional KD forces the student model to trust the teacher's soft labels equally across all images. However, medical images often contain regions of high uncertainty. The proposed controller takes the teacher’s confidence and the image’s inherent uncertainty as inputs. By applying overlapping fuzzy membership rules, it outputs a dynamic scaling weight for the KD loss. This ensures the student relies on the teacher when confidence is high and uncertainty is low, while reverting to ground-truth labels when the teacher's signal is ambiguous.
Model Selection and Image Fusion
To establish a robust teacher-student pair, the authors selected a Vision Transformer (ViT-B32) as the teacher for its global context extraction capabilities. A Genetic Algorithm (GA) was employed to screen 12 candidate CNNs, identifying MobileNet as the optimal student for its efficiency.
Before training, images undergo a pixel-level fusion process. Parallel enhancements—Gamma Correction for mid-tone adjustment and Histogram Equalization for contrast—are fused using a Discrete Wavelet Transform (wavedec2). This multi-scale approach preserves both spatial details and frequency features, providing a cleaner signal for the networks.
Experimental Setup and Datasets
The framework was validated across two distinct imaging modalities to ensure cross-domain robustness:
- LC25000 Dataset: 15,000 histopathological images across three classes (adenocarcinoma, squamous cell carcinoma, and benign tissue).
- IQ-OTH/NCCD Dataset: 1,097 CT-scan images categorized into benign, malignant, and normal cases.
Results and Discussion
FuzzyDistillViT-MobileNet achieved exceptional results, reaching 99.16% accuracy on the LC25000 dataset and 99.54% accuracy on the IQ-OTH/NCCD dataset. These metrics represent a significant improvement over standard MobileNet variants and other state-of-the-art (SOTA) approaches.
As shown in the comparative analysis, the proposed method addresses the limitations of previous works—such as high computational intensity or reliance on static teacher models—by providing a lightweight yet highly accurate alternative. Ablation studies further confirmed that the fuzzy logic adjustment and the dual-enhancement image fusion were critical to these performance gains. To demonstrate real-world utility, the model was successfully deployed as an Android application, providing real-time diagnostic support on edge devices.
Conclusion
By replacing static distillation weights with a fuzzy logic-driven adaptive mechanism, this research successfully bridges the gap between high-accuracy ViTs and efficient mobile deployment. The FuzzyDistillViT-MobileNet model provides a robust, interpretable, and computationally efficient tool for lung cancer detection. Future work will focus on expanding the system to multi-modal data and further optimizing performance for devices with extremely limited hardware specifications.
Summary updated