Abstract
Predictive machine learning models nowadays are often updated in a stateless and expensive way. The two main future trends for companies that want to build machine learning-based applications and systems are real-time inference and continual updating. Unfortunately, both trends require a mature infrastructure that is hard and costly to realize on-premise. This paper defines a novel software service and model delivery infrastructure termed Continual Learning-as-a-Service (CLaaS) to address these issues. Specifically, it embraces continual machine learning and continuous integration techniques. It provides support for model updating and validation tools for data scientists without an on-premise solution and in an efficient, stateful and easy-to-use manner. Finally, this CL model service is easy to encapsulate in any machine learning infrastructure or cloud system. This paper presents the design and implementation of a CLaaS instantiation, called LiquidBrain, evaluated in two real-world scenarios. The former is a robotic object recognition setting using the CORe50 dataset while the latter is a named category and attribute prediction using the DeepFashion-C dataset in the fashion domain. Our preliminary results suggest the usability and efficiency of the Continual Learning model services and the effectiveness of the solution in addressing real-world use-cases regardless of where the computation happens in the continuum Edge-Cloud.
Introduction
In traditional production environments, machine learning models are typically updated in a stateless manner. When new data arrives, the model is retrained from scratch using the entire accumulated dataset. This "cumulative" approach is computationally expensive, slow, and requires significant infrastructure as data volume grows. Furthermore, real-world data is often non-stationary, suffering from concept drift where underlying patterns change over time—such as shifting fashion trends or varying lighting conditions in robotic sensing.
While modern MLOps tools excel at deployment and monitoring, they often struggle to handle these dynamic shifts efficiently. To bridge the gap between academic Continual Learning (CL) research and industrial-scale deployment, we propose Continual-Learning-as-a-Service (CLaaS). CLaaS is a novel infrastructure paradigm that allows models to adapt to new data on-the-fly without exhaustive retraining.
The CLaaS Paradigm and Architecture
The core of CLaaS is stateful training. Instead of discarding previous knowledge, the system continues training the existing model on new data batches. We present LiquidBrain, a concrete implementation of this paradigm.
The architecture is designed to be modular and easy to integrate into existing cloud or edge systems. It consists of three primary layers:
- Interface: A Flask-based REST API and SDK that enables data scientists to trigger updates or evaluations on-demand.
- Middleware: Manages model versioning and monitoring, providing a proxy for external storage to decouple logic from specific cloud providers.
- CL-Backend: Powered by the Avalanche library, which implements various CL algorithms and evaluation protocols.
To mitigate "catastrophic forgetting"—where a model loses past knowledge while learning new tasks—CLaaS utilizes Replay strategies. This involves maintaining a small buffer of representative past samples that are mixed with new data during model updates.
Experimental Evaluation
We evaluated LiquidBrain on two real-world benchmarks:
- CORe50: A Computer Vision benchmark for robotics, focusing on object recognition across different sessions and environments.
- DeepFashion-C: A dataset for category and attribute prediction in fashion, adapted to a "New Classes" (Class-Incremental) scenario to simulate a stream of new clothing items.
Key Results
Our empirical results demonstrate a significant efficiency trade-off:
- Efficiency Gains: In CORe50 experiments, the CL approach was 6x more efficient in time and memory compared to cumulative retraining. In the DeepFashion-C scenario, efficiency improved by 8x.
- Predictable Training Time: Unlike cumulative retraining, where time grows linearly with data size, CLaaS training time remains relatively constant, facilitating better production scheduling.
- Performance Retention: While there is a slight accuracy decay compared to the theoretical upper bound of full retraining, the Replay strategy successfully maintained high performance across 9-10 successive data experiences.
Conclusion and Future Work
CLaaS offers a scalable, efficient alternative to traditional stateless model updates. By treating computation as a finite resource and prioritizing "good enough" accuracy with massive resource savings, it enables on-demand adaptation in dynamic environments.
Future work will focus on automating hyperparameter optimization (HPO) in streaming contexts, developing more sophisticated drift detection methods to trigger updates, and testing the architecture's scalability across complex edge-to-cloud continuums.
Summary updated