PyXAB - A Python Library for \mathcal{X}-Armed Bandit and Online Blackbox Optimization Algorithms

Wenjie Li, Haoze Li, Qifan Song, Jean HonorioJ. Open Source Softw 2024 · DOI · DBLP

Abstract

PyXAB is an open-source Python library designed to bridge the gap between theoretical research and practical application in $\mathcal{X}$-armed bandit and online blackbox optimization. By providing a modular, object-oriented interface for over 10 state-of-the-art algorithms, PyXAB standardizes complex hierarchical partitioning methods. This paper introduces the library's architecture, supported algorithms, and its utility in tasks such as hyperparameter tuning and neural architecture search.

Introduction

In machine learning and engineering, optimizing "blackbox" functions—where the internal mechanics are unknown and only input-output pairs are observable—is a critical challenge. The $\mathcal{X}$-armed bandit framework, also known as Lipschitz bandits, offers a powerful approach to this problem. Unlike traditional multi-armed bandits that choose from a discrete set of options, $\mathcal{X}$-armed bandits navigate continuous or complex parameter spaces by hierarchically partitioning the domain into increasingly refined regions.

Despite their theoretical robustness, these algorithms are notoriously difficult to implement due to the need for managing complex tree structures, custom node-splitting rules, and delicate exploration-exploitation balances. Historically, implementations have been fragmented across various programming languages or remained unreleased, hindering fair comparison and production deployment. PyXAB addresses this "open-source gap" by providing a comprehensive, Pythonic suite of these tools.

The PyXAB Architecture

PyXAB is built on a modular, object-oriented design that follows the standard bandit learning paradigm. This structure allows for high extensibility and ease of use.

The modular architecture of PyXAB, illustrating the interaction between Algorithms, Partitions, Nodes, and Objectives.

The library is organized around four primary components:

  1. Algorithm: An abstract base class implementing the pull() and receive_reward() loop. It includes implementations for over 10 algorithms, including classic methods like HOO and StoSOO, as well as modern state-of-the-art techniques like GPO and VHCT.
  2. Partition: This component manages the division of the search space. Users can utilize built-in strategies such as BinaryPartition or DimensionBinaryPartition, or define custom splitting rules.
  3. Node: These serve as the fundamental units of the hierarchical tree. PyXAB allows algorithms to store specific metadata, such as confidence bounds or visitation counts, directly within these nodes.
  4. Objective: A standardized interface for defining the function to be optimized, facilitating seamless switching between different optimization tasks.

Supported Algorithms and Evaluation

PyXAB provides the first comprehensive "one-stop shop" for $\mathcal{X}$-armed bandit algorithms in Python. It covers both stochastic (noisy) and deterministic settings, targeting different regret measures (cumulative vs. simple regret). Many algorithms, such as HCT and VHCT, previously lacked public Python implementations.

To evaluate and stress-test these algorithms, the library includes several synthetic objective functions commonly used in optimization research:

  • Garland: A function with multiple local optima.
  • DoubleSine: Features complex periodic topologies.
  • Himmelblau: A well-known multi-modal function used to test an algorithm's ability to find global maxima without becoming trapped.

Key Insights and Software Standards

The primary innovation of PyXAB lies in the standardization of complexity. By abstracting the underlying hierarchical tree logic, the library transforms complex research implementations into accessible, scikit-learn-style estimators. This decoupling of Partition and Node logic from the Algorithm math allows researchers to experiment with new search space splitting strategies without re-implementing the core optimization logic.

Furthermore, PyXAB adheres to high software engineering standards to ensure it is production-ready. It maintains over 99% test coverage, follows PEP8 compliance, and provides extensive documentation via ReadTheDocs. A complete optimization loop—from domain definition to running 1,000 iterations—can typically be implemented in fewer than ten lines of code.

Limitations and Future Directions

While PyXAB provides a robust foundation, the field of online blackbox optimization is rapidly evolving. The library is designed to be community-driven, with an architecture that encourages the addition of new algorithms. Currently, practitioners must write minor wrapper code to connect PyXAB's pull/receive_reward loop to specific external frameworks (e.g., PyTorch or TensorFlow). Future work will focus on deeper ecosystem integrations to further simplify real-world deployment for hyperparameter tuning and neural architecture search.

Summary updated

BibTeX
@article{li2024pyxab,
  author = {Wenjie Li and Haoze Li and Qifan Song and Jean Honorio and Qifan Song and Haoze Li and Wenjie Li},
  title = {{PyXAB - A Python Library for \textbackslash\{\}mathcal\{X\}-Armed Bandit and Online Blackbox Optimization Algorithms}},
  journal = {J. Open Source Softw.},
  year = {2024},
  doi = {10.21105/joss.06507},
}

Loading related content...

Figure