Abstract
This paper proposes a novel genetic design for an optimal classifier ensemble for vehicle detection using a Genetic Algorithm with Varying Population Size (GAVaPS). Recently, ensembles have been increasingly used to handle massive datasets; however, the search space for the optimal subset of classifiers grows exponentially with the size of the pool. To address this, we employ GAVaPS, which outperforms the Simple Genetic Algorithm (SGA) in efficiency. Experimental results demonstrate that the proposed method achieves high recognition accuracy while significantly reducing computational overhead.
Introduction
In pattern recognition, classifier ensembles—groups of multiple models working in concert—typically provide superior performance compared to single models. However, identifying the optimal subset of classifiers from a large pool is a significant combinatorial challenge. For a pool of $D$ potential classifiers, there are $2^D$ possible combinations. As the pool size increases to accommodate complex data, standard optimization techniques like the Simple Genetic Algorithm (SGA) often struggle with the "curse of dimensionality," leading to slow convergence or entrapment in local optima due to their fixed population sizes.
Methodology
To navigate this expansive search space, the researchers implemented GAVaPS (Genetic Algorithm with Varying Population Size). Unlike standard GAs, GAVaPS introduces a dynamic population mechanism based on the concept of "lifespan."
The GAVaPS Mechanism
- Fitness-Based Longevity: Each candidate solution (chromosome) is assigned a lifespan based on its fitness. High-performing solutions live longer, allowing them more opportunities to reproduce, while poor solutions are removed quickly.
- Dynamic Population: This allows the population size to fluctuate naturally, focusing computational resources on the most promising regions of the search space rather than maintaining a fixed, potentially inefficient population.
Ensemble Encoding
The ensemble is represented as a binary string where each bit corresponds to a classifier in the pool. A value of '1' indicates the classifier is included in the ensemble, while '0' indicates exclusion.
The fitness function evaluates candidates by balancing high recognition accuracy on a tuning set with a penalty for the number of classifiers used, thereby favoring leaner, more efficient ensembles.
Experimental Setup
The study utilized the Intelligent Vehicle Database, consisting of 120 images (60 car images and 60 non-car images).
Images were preprocessed to 50x50 pixels in grayscale. Features were extracted using Principal Component Analysis (PCA), resulting in 2,000 features. The base classifier pool consisted of 100 K-Nearest Neighbor (KNN) classifiers.
Key Results and Discussion
The performance of GAVaPS was compared against a Simple Genetic Algorithm (SGA) across several key metrics:
- Computational Speedup: GAVaPS demonstrated a dramatic improvement in efficiency. To reach 100 generations, the SGA required 69,112 minutes, whereas GAVaPS completed the task in just 7,199 minutes—a ~9.6x speedup.
- Recognition Accuracy: Despite the reduced computation time, GAVaPS maintained a maximum recognition rate of 93%, matching the SGA, and achieved a slightly higher average recognition rate (91% vs. 90%).
- Search Efficiency: GAVaPS required significantly fewer individual fitness evaluations to find the optimal ensemble, proving to be a more surgical tool for high-dimensional optimization.
Conclusion
The GAVaPS approach successfully addresses the computational bottleneck of ensemble design for vehicle detection. By allowing for dynamic population sizes and fitness-based attrition, the algorithm avoids redundant calculations and premature convergence. Future work should focus on validating this framework with larger, more diverse datasets and exploring the integration of heterogeneous base classifiers beyond KNN models.
Summary updated