Fine-grained Image Classification using Convolutional Neural Network and Support Vector Machine

Yu Shi, Tao Lin, Wei He, Biao Chen, Ruixia Wang, Nan Jiang, Yabo ZhangJournal of Networking and Network Applications 2022 · DOI

Abstract

Although vast neural network models can classify images, sub-classification for each image class requires retraining the existing model with new fine-grained data, subject to high training cost and uncertain classification accuracy. To solve the aforementioned problems, a method using Convolution Neural Network and Support Vector Machine is proposed, where the former extracts general features from images for sub-classification while the latter categorizes these features. The method can be rapidly deployed in most Internet of Things systems to identify various targets; its effectiveness in reducing training cost and improving classification accuracy is verified through experiments.

Introduction

Standard image classification is typically "coarse-grained," focusing on broad categories such as distinguishing a dog from a cat. However, "fine-grained" classification—distinguishing between sub-classes like a Husky and a Malamute—presents a significantly higher challenge. Sub-classes often share nearly identical visual features, while environmental factors like lighting, posture, and background noise further complicate the task.

Currently, adding new sub-classes to a deep learning model often requires retraining the entire network or modifying its architecture, which is computationally expensive and time-consuming. This paper proposes a hybrid architecture that decouples feature extraction from classification, creating a reusable pipeline where a general-purpose Convolutional Neural Network (CNN) handles feature extraction and a lightweight Support Vector Machine (SVM) handles specific sub-classification tasks.

Methodology

The proposed system utilizes a two-component stack designed for efficiency and accuracy:

  1. The Feature Extractor (CNN): A simplified VGG-like architecture is employed. Instead of using a final Softmax layer for classification, the network serves as a dimensionality reduction tool. It compresses high-dimensional input samples (e.g., 12,288 dimensions) into compact 256-dimensional feature vectors. This approach is superior to traditional methods like PCA because the CNN learns spatial hierarchies relevant to the image content.
  2. The Classifier (SVM Tree): These feature vectors are fed into a multi-tier SVM structure. The authors employ a "one-rest" strategy, where multiple binary SVMs are trained to distinguish one specific class from others, allowing for efficient multi-label classification.
The proposed hybrid architecture combining a VGG-based feature extractor with an SVM classifier for fine-grained output.

To optimize the CNN, the authors implemented Batch Normalization (BN) to accelerate convergence and PReLu (Parametric Rectified Linear Unit) as the activation function to improve robustness. Data augmentation techniques, such as rotating and flipping images, were also used to prevent overfitting.

Experimental Setup

The study utilized the PubFig dataset, a public face dataset containing over 58,000 images of 200 individuals. The primary task was fine-grained face classification, specifically identifying attributes like the presence of glasses or a beard. The data was divided into an 8:2 ratio for training and testing respectively.

Results and Discussion

The hybrid CNN-SVM approach significantly outperformed standalone models and traditional boosting methods. The system achieved a fine-grained classification accuracy of 96.5%, compared to 75% for a standalone simplified CNN and 82.8% for a standalone SVM. It also surpassed traditional boosting methods (PCA + LDA + SVM + Adaboost), which reached 94.79%.

Accuracy curves demonstrating the impact of data enhancement and batch normalization on model training and convergence.

A key insight from the experiments was the impact of Batch Normalization. As shown in the training curves, adding BN layers not only reduced overfitting but also significantly accelerated convergence, reaching stability in approximately 400 rounds compared to much longer durations without it. This efficiency makes the model ideal for Internet of Things (IoT) terminals where computational power is limited.

Conclusion

By decoupling "seeing" (feature extraction) from "deciding" (classification), this research demonstrates that a CNN's learned knowledge can be effectively repurposed for fine-grained tasks using an SVM head. This hybrid model reduces training costs while improving accuracy. Future work will explore substituting the current backbone with deeper architectures like ResNet and deploying the system in real-world scenarios such as the Internet of Vehicles and public health monitoring for mask detection.

Summary updated

Abstract

Although vast neural network models can classify images, sub-classification for each image class requires retraining the exisiting model with new fine-grained data, subject to high training cost and uncertain classification accuracy. To solve the aforementioned problems, a method using Convolution Neural Network and Support Vector Machine is proposed, where the former extracts general features from images for sub-classification while the latter categorizes these features. The method can be rapidly deployed in most Internet of Things systems to identify various targets; its effectiveness in reducing training cost and improving classification accuracy is verified through experiments.
BibTeX
@article{shi2022fine,
  author = {Yu Shi and Tao Lin and Wei He and Biao Chen and Ruixia Wang and Nan Jiang and Yabo Zhang},
  title = {{Fine-grained Image Classification using Convolutional Neural Network and Support Vector Machine}},
  journal = {Journal of Networking and Network Applications},
  year = {2022},
  doi = {10.33969/j-nana.2022.020204},
}
Figure