Thyroid Nodule Ultrasound Image Classification Through Hybrid Feature Cropping Network

Ruoning Song, Long Zhang, Chuang Zhu, Jun Liu, Jie Yang, Tong ZhangIEEE Access 2020 · OpenReview

Abstract

With the increasing cases of thyroid malignant tumors, the diagnosis of thyroid nodules has attracted more and more attention. Deep learning has achieved promising results in computer-aided diagnosis due to the advantages of obtaining high-dimensional features. In this paper, we proposed a hybrid multi-branch convolutional neural network based on feature cropping method for feature extraction and classification of thyroid nodule ultrasound images. Firstly, we designed a backbone convolutional neural network to extract shared feature maps and a classification network as a global branch. Next, we added a feature cropping branch in the network to perform multi-cropping on batch feature maps, to reduce the impact on classification caused by the similarity of local features between benign and malignant thyroid nodule images. Finally, based on softmax predictions of different branch feature maps, we utilize a weighted cross-entropy loss function to train our proposed binary-classification network. Experimental results show that our proposed method has achieved 96.13% accuracy, 93.24% precision, 97.18% recall, and 95.17% F1-measure in public and local datasets, outperforming other models.

Introduction

Thyroid nodules are a prevalent clinical finding, appearing in over 60% of the population during their lifetime. While the majority of these nodules are benign, early and accurate identification of the malignant minority is critical for patient survival and effective treatment planning. Ultrasound remains the primary non-invasive screening tool; however, distinguishing between benign and malignant nodules is a complex task for both clinicians and standard computer-aided diagnosis (CAD) systems.

The primary challenge lies in "local similarity." As shown in Figure 1, benign and malignant nodules often exhibit nearly identical visual characteristics in specific regions. Standard convolutional neural networks (CNNs) often overlook the subtle, localized textures or edge characteristics—such as irregular margins—that are diagnostic of malignancy. Furthermore, traditional data augmentation techniques like image-level cropping are computationally expensive, as the model must process each crop as a separate input.

Comparison of ultrasound images for (a) Benign Thyroid Nodules and (b) Malignant Thyroid Nodules, illustrating the challenge of local similarity.

Methodology

To address these challenges, we propose a multi-branch architecture that emphasizes efficiency and local detail through a hybrid feature cropping approach.

1. Backbone and Global Branch

The system utilizes a lightweight 4-layer CNN as a backbone to extract a global feature map from the entire ultrasound image. This global branch provides the context of the entire nodule and surrounding tissue.

2. Feature Cropping Branch

Unlike traditional methods that crop the raw input image, our network performs cropping on the intermediate feature maps generated by the backbone. This allows for shared computation: the backbone runs only once, and the sub-branches analyze different "windows" of the resulting high-dimensional data. We explored two strategies:

  • Random Feature Cropping (RFC): Crops are taken from random locations within the feature map.
  • Boundary Feature Cropping (BFC): Crops are specifically targeted at the margins of the nodule. Radiologists consider the contour and margin of a nodule to be key indicators of malignancy, and BFC allows the model to focus on these critical areas.

3. Weighted Loss Function and Fusion

In medical diagnostics, a false negative (missing a cancer) is significantly more detrimental than a false positive. We implemented a weighted cross-entropy loss function that penalizes the misclassification of malignant nodules more heavily. The final classification is a weighted fusion of the predictions from the global branch and the local cropped branches.

Experimental Results

The model was evaluated using the public DDTI (Digital Database of Thyroid Ultrasound Images) and a local dataset of 377 images provided by the Ultrasonic Medical Center of The First People’s Hospital of Chenzhou.

Our proposed hybrid network (Global + BFC) significantly outperformed industry-standard architectures like ResNet50 and VGG19. Key performance metrics include:

  • Accuracy: 96.13%
  • Recall: 97.18% (ensuring minimal missed diagnoses)
  • Precision: 93.24%
  • F1-Measure: 95.17%

Interestingly, the custom 4-layer backbone outperformed deeper models like Inception_v3. This suggests that for relatively small medical datasets, massive models may overfit to noise, whereas a more streamlined architecture can better capture the underlying pathology.

Conclusion

The Hybrid Feature Cropping Network demonstrates that focusing on marginal regions through feature-level analysis provides a highly discriminative tool for thyroid nodule classification. By reducing computational overhead and prioritizing clinically relevant features (the nodule boundary), the model achieves state-of-the-art performance. Future work will explore the integration of attention mechanisms to automatically identify suspicious regions and the use of weakly supervised learning to leverage larger, unannotated datasets.

Summary updated

Abstract

With the increasing cases of thyroid malignant tumors, the diagnosis of thyroid nodule has attracted more and more attention. Deep learning has achieved promising results in computer-aided diagnosis due to the advantages of obtaining high-dimensional features. In this paper, we proposed a hybrid multi-branch convolutional neural network based on feature cropping method for feature extraction and classification of thyroid nodule ultrasound images. Firstly, we designed a backbone convolutional neural network to extract shared feature maps and a classification network as global branch. Next, we added a feature cropping branch in the network to perform multi-cropping on batch feature maps, to reduce the impact on classification caused by the similarity of local features between benign and malignant thyroid nodule images. Finally, based on softmax predictions of different branch feature maps, we utilize a weighted cross-entropy loss function to train our proposed binary-classification network. Experimental results show that our proposed method has achieved 96.13% accuracy, 93.24% precision, 97.18% recall, and 95.17% F1-measure in public dataset and local dataset, outperforming other models.
BibTeX
@inproceedings{song2020thyroid,
  author = {Ruoning Song and Long Zhang and Chuang Zhu and Jun Liu and Jie Yang and Tong Zhang},
  title = {{Thyroid Nodule Ultrasound Image Classification Through Hybrid Feature Cropping Network}},
  booktitle = {IEEE Access},
  year = {2020},
  url = {https://openreview.net/forum?id=mIViiRvLCJ},
}
Figure