Study on a Novel Data Classification Method Based on Improved GA and SVM Model

Jing Huo, Yu‐Xiang ZhaoInternational Journal of Smart Home 2016 · DOI

TL;DR

This paper introduces an improved Genetic Algorithm (GA) to optimize Support Vector Machine (SVM) parameters, leading to a novel classification method called IGASVM. The proposed method aims to enhance SVM's generalization ability and classification accuracy, particularly for complex datasets. Experimental results demonstrate that IGASVM outperforms standard SVM and GA-SVM approaches.

What Problem Does This Paper Address?

In the era of big data, extracting meaningful insights from vast amounts of information is a significant challenge. Data classification is a crucial technique for this, and while Support Vector Machines (SVMs) are powerful for handling complex datasets (small samples, nonlinearities, high dimensions), they suffer from limitations like weak generalization ability and suboptimal classification accuracy. A key issue with SVMs is the difficulty in selecting appropriate kernel functions and optimizing their parameters. This paper tackles these limitations by proposing an enhanced approach to SVM parameter optimization.

Methods

The core of this work is the development of an Improved Genetic Algorithm (IGA) designed to optimize SVM parameters. The IGA incorporates two key strategies:

  • Self-Adaptive Control Parameter Strategy: This strategy aims to maintain population diversity within the GA, detect premature convergence early, and help the algorithm escape local optima.
  • Improving Convergence Speed Strategy: This strategy further refines the GA's search process to enhance its efficiency.

The improved GA is then used to tune the crucial parameters of the SVM model, specifically the kernel function parameters and the penalty factor (often denoted as C). This optimized SVM, termed IGASVM, is then applied to classification tasks. The paper also details the mathematical formulation of SVM, including its optimization problem in both primal and dual spaces, and the common kernel functions, with a focus on the Radial Basis Function (RBF) kernel.

Datasets

The experiments were conducted using several publicly available datasets from the UCI Machine Learning Repository. These include:

  • Wine
  • Pittsburgh Bridges
  • Balance Scale
  • Libras Movement
  • Arrhythmia
  • Spectrometer

The paper provides a table (Table 1) detailing the number of samples and attributes for each dataset. No new benchmark datasets were introduced.

Key Insights and Ideas

  • GA's Potential for SVM Optimization: The paper highlights that while GA is a powerful optimization tool, its standard implementation can struggle with premature convergence and local optima. The proposed improvements to GA directly address these weaknesses.
  • Parameter Sensitivity of SVM: The research underscores that the performance of SVM is highly sensitive to the choice of kernel function and its parameters, as well as the penalty factor. Effective optimization of these parameters is critical for achieving good generalization and accuracy.
  • Synergy of IGA and SVM: The combination of an improved GA with SVM creates a more robust classification system. The IGA's ability to explore the parameter space effectively helps overcome SVM's inherent challenges in parameter selection.
  • Improved Generalization: By optimizing SVM parameters, the IGASVM method aims to find a better balance between fitting the training data and generalizing to unseen data, thus reducing overfitting.

Key Results

The IGASVM method demonstrated superior performance compared to standard SVM and a basic GA-SVM approach across all tested datasets. Specifically, the paper reports the following classification accuracies:

  • Wine: SVM (81.73%), GA-SVM (88.62%), IGASVM (93.65%)
  • Pittsburgh Bridges: SVM (82.95%), GA-SVM (85.36%), IGASVM (94.79%)
  • Balance Scale: SVM (83.68%), GA-SVM (89.17%), IGASVM (90.34%)
  • Libras Movement: SVM (76.81%), GA-SVM (83.72%), IGASVM (91.48%)
  • Arrhythmia: SVM (74.21%), GA-SVM (78.63%), IGASVM (90.65%)
  • Spectrometer: SVM (82.36%), GA-SVM (85.94%), IGASVM (91.13%)

The authors conclude that the improved GA effectively optimizes SVM parameters, leading to better learning ability, generalization ability, and overall classification accuracy for the IGASVM method.

Limitations and Future Work

  • Parameter Tuning for IGA: While the paper introduces strategies to improve GA, the initial parameters for the IGA itself (e.g., population size, mutation probability) were obtained through continuous testing, suggesting that these might still require careful tuning.
  • Computational Cost: Optimizing SVM parameters with GA, even an improved one, can be computationally intensive. The paper mentions improving speed and efficiency, but a detailed analysis of the computational overhead compared to other methods is not extensively discussed.
  • Generalizability: The study focuses on specific datasets. Further validation on a wider range of diverse and complex datasets would strengthen the claims of IGASVM's broad applicability.

The authors suggest that the improved GA can effectively optimize and determine SVM parameters, leading to better learning and generalization abilities. Future work could explore applying this method to other complex classification problems and potentially investigate more advanced optimization techniques for the GA itself.

Summary updated

Abstract

Support vector machine(SVM) can effectively solve the classification problem with small samples, nonlinear and high dimensions, but it exits the weak generalization ability and low classification accuracy. So an improved genetic algorithm(IGA) is introduced in order to propose a new classification(IGASVM) method based on combining improved GA and SVM model. In the proposed IGASVM method, the self-adaptive control parameter strategy and improving convergence speed strategy are introduced into the GA to keep the diversity of the population, promptly reflect the premature convergence of the individual and escape from the local optimal solution for improving the search performance. Then the improved GA is used to optimize and determine the parameters of the SVM model in order to improve the learning ability and generalization ability of the SVM model for obtaining new classification (IGASVM) method. Finally, the experiment data is selected to test the effectiveness of the proposed IGASVM method. The experiment results show that the improved GA can effectively optimize and determine the parameters of the SVM model, and the IGASVM method takes on the better learning ability, generalization ability and classification accuracy.
BibTeX
@article{huo2016study,
  author = {Jing Huo and Yu‐Xiang Zhao},
  title = {{Study on a Novel Data Classification Method Based on Improved GA and SVM Model}},
  journal = {International Journal of Smart Home},
  year = {2016},
  doi = {10.14257/ijsh.2016.10.5.12},
}
Figure