Comparing LBP, HOG and Deep Features for Classification of Histopathology Images

Taha J. Alhindi, Shivam Kalra, Ka Hin Ng, Anika Afrin, Hamid R. Tizhoosh

Abstract

This study evaluates the effectiveness of traditional handcrafted features—Local Binary Patterns (LBP) and Histogram of Oriented Gradients (HOG)—against deep features extracted from a pre-trained VGG19 network for the classification of histopathology images. Using the KIMIA Path960 dataset, we compare various feature extractors paired with Support Vector Machines (SVM), Decision Trees, and Artificial Neural Networks. Our results demonstrate that LBP features combined with an SVM achieve a superior accuracy of 90.52%, significantly outperforming off-the-shelf deep features (81.14%). These findings highlight the critical role of local texture descriptors in medical imaging, where a significant domain gap exists between natural images and cellular structures.

Introduction

Histopathology, the microscopic examination of tissue to study the manifestations of disease, is fundamental to cancer diagnosis. While deep learning has become the standard in computer vision, its application to medical imaging is often hindered by the "domain gap." Most state-of-the-art models are pre-trained on ImageNet, which contains distinct objects like animals or vehicles. In contrast, histopathology images are characterized by complex, repetitive, and stochastic textures of cells and stroma. This paper investigates whether modern deep features are truly superior to traditional texture-based descriptors for identifying tissue types in medical scans.

Methodology

The research evaluates three distinct feature extraction approaches:

  1. Local Binary Patterns (LBP): A classic texture descriptor that labels pixels by thresholding the neighborhood and converting the result into a binary number. This method is highly effective at capturing local spatial patterns.
  2. Histogram of Oriented Gradients (HOG): A descriptor that counts occurrences of gradient orientations in localized portions of an image, typically used for shape and edge detection.
  3. Deep Features (VGG19): Features extracted from a VGG19 model pre-trained on ImageNet. To identify the optimal level of abstraction, features were extracted from four different depths: fc1, block5_pool, block4_pool, and block3_pool.
Flowchart of the Local Binary Pattern (LBP) feature extraction process, which proved to be the most effective method in this study.

These features were processed using three classification algorithms: Support Vector Machines (SVM) with an RBF kernel, Decision Trees (DT), and shallow Artificial Neural Networks (ANN). Robustness was ensured through 3-fold cross-validation.

Dataset

The study utilized the KIMIA Path960 dataset, a publicly available benchmark consisting of 960 histopathology images. These images represent 20 different tissue classes, each down-sampled to 308 × 168 pixels. The dataset is particularly challenging due to the high visual similarity between different tissue types, requiring descriptors that can distinguish subtle textural nuances.

A grid representing the 20 different tissue classes in the KIMIA Path960 dataset, illustrating the complex and repetitive textures inherent in histopathology.

Results and Discussion

The experimental results revealed a clear hierarchy in performance:

  • Superiority of LBP: The LBP features paired with an SVM achieved the highest overall accuracy of 90.52%. This suggests that local texture information is more discriminative for histopathology than the high-level "object" shapes deep networks are typically optimized to find.
  • Deep Feature "Sweet Spot": Among the VGG19 features, the best performance (81.14%) was achieved at the block4_pool layer. Accuracy dropped significantly at the final fc1 layer (44.79%) and the earlier block3_pool layer (67.49%). This indicates that as networks go deeper into ImageNet-specific features, they lose the generality required for medical textures.
  • HOG Failure: HOG performed poorly, peaking at only 36.15% accuracy. Its reliance on edge-orientation is ill-suited for the "blobby" and irregular nature of cellular structures.

Conclusion

Traditional handcrafted features, specifically LBP, remain highly relevant and effective for histopathology image classification. While deep learning offers immense potential, off-the-shelf models pre-trained on natural images may not always outperform specialized texture descriptors. Future work should focus on fine-tuning deep models specifically on medical datasets or developing hybrid models that fuse the local precision of LBP with the hierarchical context of deep neural networks.

Summary updated

Abstract

Medical image analysis has become a topic under the spotlight in recent\nyears. There is a significant progress in medical image research concerning the\nusage of machine learning. However, there are still numerous questions and\nproblems awaiting answers and solutions, respectively. In the present study,\ncomparison of three classification models is conducted using features extracted\nusing local binary patterns, the histogram of gradients, and a pre-trained deep\nnetwork. Three common image classification methods, including support vector\nmachines, decision trees, and artificial neural networks are used to classify\nfeature vectors obtained by different feature extractors. We use KIMIA Path960,\na publicly available dataset of $960$ histopathology images extracted from $20$\ndifferent tissue scans to test the accuracy of classification and feature\nextractions models used in the study, specifically for the histopathology\nimages. SVM achieves the highest accuracy of $90.52\\%$ using local binary\npatterns as features which surpasses the accuracy obtained by deep features,\nnamely $81.14\\%$.\n
BibTeX
@inproceedings{alhindi2018comparing,
  author = {Taha J. Alhindi and Shivam Kalra and Ka Hin Ng and Anika Afrin and Hamid R. Tizhoosh},
  title = {{Comparing LBP, HOG and Deep Features for Classification of Histopathology Images}},
  booktitle = {IJCNN},
  year = {2018},
  doi = {10.1109/ijcnn.2018.8489329},
}
Figure