SuBiTO: Synopsis-based Training Optimization for Continuous Real-Time Neural Learning over Big Streaming Data

Errikos Streviniotis, George Klioumis, Nikos GiatrakosAAAI 2025 · DOI

TL;DR

This paper introduces SuBiTO, a framework for optimizing neural network training in real-time streaming data scenarios. SuBiTO automatically balances the trade-off between training speed and model accuracy by dynamically adjusting neural network architecture, data synopsis parameters, and training epochs, enabling continuous learning without sacrificing performance.

What Problem Does This Paper Address?

In many real-time applications like content moderation or anomaly detection, neural networks (NNs) need to be continuously trained and updated on massive, high-velocity data streams. While complex NNs and larger datasets can improve accuracy, they significantly increase training time, potentially making real-time predictions impossible. Existing methods for speeding up training often approximate matrix products but don't provide a clear understanding of the resulting accuracy-time trade-offs. This leaves a critical gap: how to achieve high accuracy in continuous learning systems while maintaining the strict latency requirements of real-time applications.

Methods

SuBiTO addresses this challenge by employing a Bayesian Optimization (BO) approach to automatically tune several key parameters:

  • NN Architecture: The framework can adjust the number, size, and type of NN layers.
  • Data Synopsis Parameters: It optimizes how data is summarized or sampled (e.g., using sketches or dimensionality reduction) to reduce the volume of data processed during training without losing essential information. This is crucial for "load shedding" in high-throughput streams.
  • Training Epochs: SuBiTO determines the optimal number of training epochs for each model update.

The core of SuBiTO involves a Training Pipeline that continuously ingests data, applies synopses, and trains an NN. A Prediction Pipeline uses the latest trained model for real-time inference. A Statistics Collector monitors training time and accuracy. When a concept drift is detected (indicating a shift in data distribution), the SuBiTO Optimizer is triggered. This optimizer uses BO to explore different parameter combinations, guided by acquisition functions (like lower confidence bound, expected improvement, etc.). A Gaussian Process Regressor (GPR) models the trade-offs, and SuBiTO suggests optimal parameter sets that balance accuracy and training time, often using a score function that penalizes exceeding a target latency.

Datasets

The paper mentions using two real-world datasets for demonstration:

  • NSFW Detect: Used for image moderation analytics.
  • UCF50: Used for human action recognition in videos.

These datasets are used to simulate both the labeled training streams and the unlabeled prediction streams, with the latter being generated by GANs or by splitting the original datasets to mimic high-volume, continuous data flow.

Key Insights and Ideas

  • Automated Trade-off Management: The most significant insight is the development of a framework that automatically learns and manages the complex trade-off between training speed and predictive accuracy in a continuous learning setting. This removes the need for manual tuning, which is impractical in dynamic streaming environments.
  • Synopsis as a Key Lever: The paper highlights the importance of data synopses not just for reducing data volume but as a tunable parameter that directly impacts the training-time-accuracy balance.
  • Bayesian Optimization for Real-time ML: Applying BO to this specific problem of continuous real-time NN optimization is a novel approach, allowing for efficient exploration of a large parameter space with limited trials.
  • Concept Drift as a Trigger: Using concept drift detection as the trigger for re-optimization ensures that the system adapts proactively when performance might degrade, maintaining relevance.

Key Results

The paper describes the SuBiTO architecture and its components, emphasizing its ability to provide a priori known balances between training time and accuracy. While specific quantitative results (e.g., percentage speedup or accuracy gains) are not detailed in the provided text, the framework is presented as a solution that:

  • Suggests optimal sets of NN layers, synopsis parameters, and training epochs.
  • Detects concept drifts.
  • Enables human operators to adapt parameters on-the-fly at runtime via a dashboard.
  • Integrates with popular frameworks like TensorFlow and PyTorch.

Limitations and Future Work

  • Score Function Flexibility: The paper notes that the specific score function used to balance accuracy and training time is orthogonal to SuBiTO and can be chosen by the user. This implies that the effectiveness of the optimization might depend on the chosen scoring metric.
  • Human-in-the-Loop: While SuBiTO suggests optimal parameters, the human operator ultimately selects the best option. This reliance on human judgment could be a bottleneck or introduce subjectivity.
  • Future Work: The paper implies that further exploration of different synopsis techniques, acquisition functions, and scoring mechanisms could enhance the framework's capabilities. The development of a user-friendly dashboard is also highlighted as a key aspect for practical deployment.

Summary updated

BibTeX
@inproceedings{streviniotis2025subito,
  author = {Errikos Streviniotis and George Klioumis and Nikos Giatrakos},
  title = {{SuBiTO: Synopsis-based Training Optimization for Continuous Real-Time Neural Learning over Big Streaming Data}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i28.35371},
}
Figure