Optimal Classifier Ensemble Design for Vehicle Detection Using GAVaPS

Heesung Lee, Jae-Hung Lee, Euntai KimJournal of Institute of Control Robotics and Systems 2010 · DOI

Abstract

This paper proposes a novel genetic design for an optimal classifier ensemble for vehicle detection using a Genetic Algorithm with Varying Population Size (GAVaPS). Recently, ensembles have been increasingly used to handle massive datasets; however, the search space for the optimal subset of classifiers grows exponentially with the size of the pool. To address this, we employ GAVaPS, which outperforms the Simple Genetic Algorithm (SGA) in efficiency. Experimental results demonstrate that the proposed method achieves high recognition accuracy while significantly reducing computational overhead.

Introduction

In pattern recognition, classifier ensembles—groups of multiple models working in concert—typically provide superior performance compared to single models. However, identifying the optimal subset of classifiers from a large pool is a significant combinatorial challenge. For a pool of $D$ potential classifiers, there are $2^D$ possible combinations. As the pool size increases to accommodate complex data, standard optimization techniques like the Simple Genetic Algorithm (SGA) often struggle with the "curse of dimensionality," leading to slow convergence or entrapment in local optima due to their fixed population sizes.

Methodology

To navigate this expansive search space, the researchers implemented GAVaPS (Genetic Algorithm with Varying Population Size). Unlike standard GAs, GAVaPS introduces a dynamic population mechanism based on the concept of "lifespan."

The GAVaPS Mechanism

  • Fitness-Based Longevity: Each candidate solution (chromosome) is assigned a lifespan based on its fitness. High-performing solutions live longer, allowing them more opportunities to reproduce, while poor solutions are removed quickly.
  • Dynamic Population: This allows the population size to fluctuate naturally, focusing computational resources on the most promising regions of the search space rather than maintaining a fixed, potentially inefficient population.

Ensemble Encoding

The ensemble is represented as a binary string where each bit corresponds to a classifier in the pool. A value of '1' indicates the classifier is included in the ensemble, while '0' indicates exclusion.

Binary chromosome structure for classifier selection.

The fitness function evaluates candidates by balancing high recognition accuracy on a tuning set with a penalty for the number of classifiers used, thereby favoring leaner, more efficient ensembles.

Experimental Setup

The study utilized the Intelligent Vehicle Database, consisting of 120 images (60 car images and 60 non-car images).

Sample vehicle images from the dataset used for training and testing.

Images were preprocessed to 50x50 pixels in grayscale. Features were extracted using Principal Component Analysis (PCA), resulting in 2,000 features. The base classifier pool consisted of 100 K-Nearest Neighbor (KNN) classifiers.

Key Results and Discussion

The performance of GAVaPS was compared against a Simple Genetic Algorithm (SGA) across several key metrics:

  • Computational Speedup: GAVaPS demonstrated a dramatic improvement in efficiency. To reach 100 generations, the SGA required 69,112 minutes, whereas GAVaPS completed the task in just 7,199 minutes—a ~9.6x speedup.
  • Recognition Accuracy: Despite the reduced computation time, GAVaPS maintained a maximum recognition rate of 93%, matching the SGA, and achieved a slightly higher average recognition rate (91% vs. 90%).
  • Search Efficiency: GAVaPS required significantly fewer individual fitness evaluations to find the optimal ensemble, proving to be a more surgical tool for high-dimensional optimization.

Conclusion

The GAVaPS approach successfully addresses the computational bottleneck of ensemble design for vehicle detection. By allowing for dynamic population sizes and fitness-based attrition, the algorithm avoids redundant calculations and premature convergence. Future work should focus on validating this framework with larger, more diverse datasets and exploring the integration of heterogeneous base classifiers beyond KNN models.

Summary updated

Abstract

This paper proposes novel genetic design of optimal classifier ensemble for vehicle detection using Genetic Algorithm with Varying Population Size (GAVaPS). Recently, many classifiers are used in classifier ensemble to deal with tremendous amounts of data. However the problem has a exponential large search space due to the increasing the number of classifier pool. To solve this problem, we employ the GAVaPS which outperforms comparison with simple genetic algorithm (SGA). Experiments are performed to demonstrate the efficiency of the proposed method.
BibTeX
@article{lee2010optimal,
  author = {Heesung Lee and Jae-Hung Lee and Euntai Kim},
  title = {{Optimal Classifier Ensemble Design for Vehicle Detection Using GAVaPS}},
  journal = {Journal of Institute of Control Robotics and Systems},
  year = {2010},
  doi = {10.5302/j.icros.2010.16.1.096},
}
Figure