Abstract
The General Automated Machine learning Assistant (GAMA) is a modular AutoML system developed to empower users to track and control how AutoML algorithms search for optimal machine learning pipelines, and facilitate AutoML research itself. In contrast to current, often black-box systems, GAMA allows users to plug in different AutoML and post-processing techniques, logs and visualizes the search process, and supports easy benchmarking. It currently features three AutoML search algorithms, two model post-processing steps, and is designed to allow for more components to be added.
Introduction
Automated Machine Learning (AutoML) has become a vital tool for finding optimal machine learning pipelines—encompassing preprocessing, model selection, and hyperparameter tuning—without exhaustive manual effort. However, many existing frameworks like auto-sklearn or TPOT are monolithic and operate as "black boxes." This lack of transparency creates barriers for researchers who need to isolate specific components and practitioners who want to understand the trade-offs made during automation.
GAMA (General Automated Machine learning Assistant) addresses these issues by providing a modular, open-source architecture. By decoupling search strategies, post-processing, and evaluation, GAMA allows for a "pluggable" system that is both flexible for experts and accessible for beginners.
System Architecture and Modularity
GAMA’s core innovation is its modular pipeline, which allows users to compose a custom AutoML system from three main stages:
1. Search Algorithms
GAMA currently supports three distinct search strategies:
- Random Search: A baseline approach for pipeline discovery.
- ASHA (Asynchronous Successive Halving): A resource-efficient method that uses "low-fidelity" estimates to quickly prune underperforming pipelines.
- Asynchronous Evolutionary Algorithm: Based on NSGA-II, this uses "survival of the fittest" to evolve pipelines. By operating asynchronously, GAMA maximizes hardware utilization, as workers do not wait for a full generation to finish before starting new evaluations.
2. Post-processing
Once the search phase is complete, GAMA can either select the Single Best pipeline discovered or build an Ensemble (using the Caruana technique) to improve overall robustness and predictive performance.
3. Visualization and Transparency
To provide visibility into the search process, GAMA includes a web-based graphical user interface (GUI). This dashboard logs the evaluation of every pipeline, visualizes convergence rates, and tracks the relationships between models in real-time.
Benchmarking and Performance
The authors evaluated GAMA using the open-source AutoML Benchmark, comparing it against established industry standards across various binary and multi-class classification datasets, such as higgs, miniboone, and covertype.
The results demonstrate that GAMA is highly competitive. In terms of AUC (Area Under the Curve) and Logloss, GAMA consistently performs within the same clusters as auto-sklearn, TPOT, and H2O AutoML. This confirms that GAMA’s modularity and architectural flexibility do not come at the cost of predictive accuracy.
Key Insights and Future Work
A major advantage of GAMA is its ease of extension; new search algorithms can often be integrated with fewer than 170 lines of code by leveraging shared utilities for logging and parallelization. While the current library features three search algorithms, future work aims to expand this selection and introduce "warm-starting" capabilities—using meta-data from previous tasks to jumpstart the search on new datasets.
By mimicking the scikit-learn API (fit, predict), GAMA remains easy to use while providing the transparency required for advanced machine learning research and development.
Summary updated