Abstract
We applied our method to the melanoma recognition task on ISIC-2019 and XJTU-MM datasets, which focus on both the texture and shape of the lesions. The experimental results on dermoscopic image recognition and pathological image recognition datasets show the proposed method outperforms the compared algorithms and prove the effectiveness of our method.
Introduction: The "Texture Bias" in Medical AI
Most modern computer vision models suffer from a significant "texture bias." Because these models are typically pre-trained on general datasets like ImageNet, they become highly proficient at recognizing local patterns (such as the fur of an animal) while often overlooking the global structure or "shape" of an object.
In the field of dermatology, this bias presents a major hurdle. The clinical diagnosis of a skin lesion frequently depends as much on its irregular borders and geometric symmetry (shape) as it does on its internal color patterns (texture). Previous attempts to address this have relied on "cascade" models—where a lesion is first segmented and then classified. However, these models are fragile: if the initial segmentation is even slightly inaccurate, the subsequent classification is likely to fail.
Methodology: The STNet Architecture
To overcome these limitations, the authors introduce STNet (Shape and Texture Network). This novel two-stream architecture mimics the human visual system's ability to process shape and texture information separately yet synergistically.
1. Texture-Biased Stream
This stream utilizes a specialized Pyramid-Grouped Convolution (PGC). Unlike standard convolutions where every input channel connects to every output, PGC varies connection density across layers. This allows the model to capture multi-scale texture details more efficiently without the computational overhead of fully dense connections.
2. Shape-Biased Stream
Supervised by a segmentation task (generating a lesion mask), this stream incorporates Deformable Convolutions (DC). Standard square convolution kernels are poorly suited for the "amoeba-like" shapes of melanoma. Deformable convolutions allow the network's sampling grid to stretch and bend, effectively "wrapping" its focus around the lesion's actual irregular borders.
3. Channel-Attention-based Feature Fusion (CAFF)
Rather than simply stacking the outputs of the two streams, the authors employed a Channel-Attention-based Feature Fusion (CAFF) mechanism. This attention module dynamically selects the most relevant features from both streams, filtering out redundant information and preventing overfitting.
4. Asymmetric Loss Function
Medical datasets are often imbalanced, with significantly more benign samples than malignant ones. STNet implements a custom Asymmetric Loss Function that weights hard-to-classify malignant samples more heavily, ensuring the model prioritizes accuracy in high-stakes detections.
Experimental Results
The model was evaluated on two primary datasets:
- ISIC-2019: A public dataset of 17,397 dermoscopic images.
- XJTU-MM: A skin pathological image dataset containing 9,098 high-resolution Regions of Interest (RoIs).
STNet consistently outperformed state-of-the-art models, including ResNeSt and ConvNeXt:
- ISIC-2019: STNet-101 achieved an Accuracy of 97.1% and an F1-score of 0.946, a marked improvement over the baseline ResNet-50 (92.5% accuracy).
- XJTU-MM: On pathological slides, the model reached an impressive 98.5% Accuracy.
Ablation studies confirmed that each component—PGC, DC, and CAFF—contributes significantly to the final performance, validating the synergy between shape and texture learning.
Conclusion
STNet demonstrates that joint learning of shape and texture provides a more robust and "brain-like" approach to medical image recognition. While the two-stream architecture is computationally more intensive than single-stream models, its superior accuracy makes it a compelling choice for high-stakes diagnostic tools. Future research will focus on knowledge distillation to compress this intelligence into more efficient, single-stream encoders for real-time applications.
Summary updated