Continual-Learning-as-a-Service (CLaaS): On-Demand Efficient Adaptation of Predictive Models

Rudy Semola, Vincenzo Lomonaco, Davide Bacciu

Abstract

In dynamic real-world environments, machine learning models frequently encounter concept drift, leading to performance degradation over time. Traditional mitigation involves "stateless retraining," which is computationally expensive and infrastructure-heavy. This paper proposes Continual-Learning-as-a-Service (CLaaS), a novel framework designed to integrate stateful Continual Learning (CL) strategies into standard MLOps pipelines. Our reference implementation, Continual Brain, demonstrates that stateful updates can achieve up to an 8x improvement in computational efficiency compared to traditional methods while maintaining competitive predictive accuracy.

Introduction

As machine learning moves from static research environments to production, the challenge of non-stationary data becomes paramount. Whether in robotics or e-commerce, the statistical properties of data evolve—a phenomenon known as concept drift. The current industry standard for addressing this is cumulative retraining, where models are periodically retrained from scratch on all available historical data. This approach is inherently inefficient and fails to leverage the stateful nature of learning.

Existing MLOps tools excel at deployment and monitoring but lack native support for stateful updates—the ability to learn from new data without re-processing the entire history or suffering from "catastrophic forgetting."

The CLaaS Framework

The researchers introduce CLaaS as a service-oriented architecture that bridges the gap between academic CL research and industrial application. The framework is designed to be "Edge-Cloud" agnostic, allowing developers to implement complex CL logic without deep domain expertise.

The CLaaS architecture manages sequential 'experiences' through a dedicated service provider layer.

The architecture, as shown in Figure 5, centers on a Service Provider that handles training, model management, evaluation, and logging. It interacts with a dynamic production environment where data arrives as a sequence of "experiences" ($e_0, e_1, ..., e_n$) over time.

Methods and Implementation

The core implementation, Continual Brain, is built upon the Avalanche library and structured into three layers:

  1. Interface: A REST API and SDK for triggering tasks.
  2. Middleware: Manages data drift monitoring, versioning, and storage interfacing.
  3. CL-Backend: Executes strategies like Replay, which maintains a small buffer of past examples to prevent forgetting during updates.
CLaaS integrates into the end-to-end ML pipeline, from raw data validation to serving endpoints.

By decoupling the CL logic from the underlying infrastructure (Figure 4), CLaaS allows for seamless integration into existing DataOps and DevOps workflows, providing a standardized way to transition from stateless to stateful model evolution.

Experimental Evaluation

The framework was evaluated using two real-world datasets:

  • CORe50: Focused on continuous object recognition for robotics across varying environments.
  • DeepFashion-C: Simulating a "New Classes" scenario in a fashion context over 10 successive data batches.

Key Results

  • Efficiency: In the CORe50 scenario, CLaaS achieved a 6x reduction in time and memory usage. In the DeepFashion-C scenario, efficiency gains reached nearly 8x.
  • Scalability: Unlike cumulative retraining, where time grows linearly with data volume, the Replay strategy maintained stable training times across all experiences.
  • Accuracy: Predictive performance remained competitive with the "Cumulative" upper bound, proving that efficiency does not necessitate a loss in model quality.

Conclusion and Future Work

CLaaS represents a significant step toward making Continual Learning a practical reality in production MLOps. By providing stateful updates as a service, it flattens the resource consumption curve associated with model maintenance. Future work will focus on expanding the library of available CL algorithms, exploring memory-free strategies for constrained edge devices, and further automating drift-triggered update cycles.

Summary updated

Abstract

Predictive machine learning models nowadays are often updated in a stateless and expensive way. The two main future trends for companies that want to build machine learning-based applications and systems are real-time inference and continual updating. Unfortunately, both trends require a mature infrastructure that is hard and costly to realize on-premise. This paper defines a novel software service and model delivery infrastructure termed Continual Learning-as-a-Service (CLaaS) to address these issues. Specifically, it embraces continual machine learning and continuous integration techniques. It provides support for model updating and validation tools for data scientists without an on-premise solution and in an efficient, stateful and easy-to-use manner. Finally, this CL model service is easy to encapsulate in any machine learning infrastructure or cloud system. This paper presents the design and implementation of a CLaaS instantiation, called LiquidBrain, evaluated in two real-world scenarios. The former is a robotic object recognition setting using the CORe50 dataset while the latter is a named category and attribute prediction using the DeepFashion-C dataset in the fashion domain. Our preliminary results suggest the usability and efficiency of the Continual Learning model services and the effectiveness of the solution in addressing real-world use-cases regardless of where the computation happens in the continuum Edge-Cloud.
BibTeX
@inproceedings{semola2022continual,
  author = {Rudy Semola and Vincenzo Lomonaco and Davide Bacciu},
  title = {{Continual-Learning-as-a-Service (CLaaS): On-Demand Efficient Adaptation of Predictive Models}},
  booktitle = {CI4PM/PAI@WCCI},
  year = {2022},
  url = {https://openreview.net/forum?id=pVVwp5obyKu},
}
Figure