PHOTON - A Python API for Rapid Machine Learning Model Development

Ramona Leenings, Nils Ralf Winter, Lucas Plagwitz, Vincent Holstein, Jan Ernsting, Jakob Steenweg, Julian Gebker, Kelvin Sarink, Daniel Emden, Dominik Grotegerd, Nils Opel, Benjamin Risse, Xiaoyi Jiang, Udo Dannlowski, Tim HahnarXiv 2020 · OpenReview

Introduction

The standard machine learning (ML) workflow—encompassing preprocessing, algorithm selection, hyperparameter optimization (HPO), and rigorous evaluation—is often fragmented. Researchers frequently navigate a disparate ecosystem of libraries like scikit-learn, PyTorch, and imbalanced-learn, each requiring distinct syntaxes and boilerplate code. This fragmentation increases the risk of "data leakage," where test set information inadvertently biases the training process, leading to over-optimistic performance estimates. While automated ML (AutoML) tools address some of these issues, they often operate as "black boxes," lacking the flexibility to incorporate critical domain-specific knowledge.

PHOTONAI is a high-level Python API designed to bridge this gap. It unifies various ML toolboxes into a single, automated framework that simplifies complex pipeline construction while strictly enforcing rigorous evaluation standards, such as nested cross-validation.

Architecture and Methods

PHOTONAI employs a modular, class-based architecture that prioritizes both flexibility and automation. The central management component is the Hyperpipe, which orchestrates the entire workflow, from data partitioning to final model training.

PHOTONAI Class Architecture

Key components include:

  • PipelineElement: A wrapper that facilitates the integration of any scikit-learn-compatible algorithm using simple keywords.
  • Automated Nested Cross-Validation: The framework inherently supports nested loops, ensuring hyperparameter tuning occurs strictly within training folds. This provides an unbiased estimate of generalization error.

Parallel and Nested Structures

A core innovation of PHOTONAI is its support for complex, multi-branch architectures through specialized elements:

  • Switch (OR-operation): Allows the HPO strategy to select the optimal algorithm from a set of candidates at a specific pipeline stage.
  • Stack (AND-operation): Horizontally concatenates the outputs of multiple algorithms, facilitating the creation of ensembles.
  • Branch: Enables nested sub-pipelines for processing different data modalities in parallel.
Parallel Pipeline Elements: Switch (A) and Stack (B)

Key Features

PHOTONAI introduces several features to streamline model development:

  • Algorithm Selection as a Hyperparameter: By utilizing the Switch element, the choice of estimator (e.g., Random Forest vs. SVM) becomes an optimizable parameter.
  • The "On/Off" Toggle: Each PipelineElement includes a test_disabled parameter, allowing the optimizer to determine if a preprocessing step (like PCA) should be included at all.
  • Standardized Model Sharing: The framework uses a .photon file format to save the entire optimized pipeline, including transformers and estimators, ensuring reproducibility and easy deployment.
  • Domain Knowledge Integration: Unlike rigid AutoML tools, PHOTONAI allows researchers to manually define pipeline structures while automating tuning and evaluation.

Case Study: Heart Failure Prediction

To demonstrate the API's utility, the authors applied PHOTONAI to a Heart Failure Prediction dataset (299 patients, 13 features). The goal was to predict mortality during a follow-up period.

  1. Baseline: Initial estimator selection yielded an F1-score of 0.75.
  2. Refinement: Incorporating class-balancing techniques (e.g., SMOTE) improved the balance between sensitivity and specificity.
  3. Final Results: The optimized PHOTONAI model achieved an F1-score of 0.746 on the test set, outperforming a previously published benchmark of 0.714 (Chicco et al., 2020).

Notably, the benchmark study likely inflated its results through improper feature selection; PHOTONAI’s nested approach provided a more realistic and ultimately higher-performing estimate.

Discussion and Future Work

While PHOTONAI excels at unifying standard ML workflows, extremely complex deep learning architectures with non-standard training protocols may still require custom implementation. Currently, custom algorithms not following the scikit-learn API require a small wrapper class.

Future developments include specialized modules for neuroimaging (MRI data), graph analysis, and omics data. The team also plans to implement automatic ensemble generation to further enhance model performance.

Summary updated

BibTeX
@misc{leenings2020photon,
  author = {Ramona Leenings and Nils Ralf Winter and Lucas Plagwitz and Vincent Holstein and Jan Ernsting and Jakob Steenweg and Julian Gebker and Kelvin Sarink and Daniel Emden and Dominik Grotegerd and Nils Opel and Benjamin Risse and Xiaoyi Jiang and Udo Dannlowski and Tim Hahn},
  title = {{PHOTON - A Python API for Rapid Machine Learning Model Development}},
  year = {2020},
  url = {https://openreview.net/forum?id=KWHdR8X7_d},
}
Figure