LifeLearner: Hardware-Aware Meta Continual Learning System for Embedded Computing Platforms

Young D. Kwon, Jagmohan Chauhan, Hong Jia, Stylianos I. Venieris, Cecilia Mascolo

Abstract

LifeLearner is the first hardware-aware meta-continual learning system designed to bring lifelong learning to resource-constrained IoT devices and microcontrollers. By combining a rehearsal-based meta-learning strategy with a novel "latent replay" compression pipeline, it achieves near-optimal accuracy while reducing memory footprints by up to 178x and energy consumption by over 90% compared to state-of-the-art methods.

Introduction

Deep learning models typically suffer from Catastrophic Forgetting (CF): when trained on new data, they abruptly "forget" previously learned information. Continual Learning (CL) aims to solve this, but existing solutions are often a poor fit for the "edge" (sensors, wearables, and robots) due to data scarcity, extreme resource constraints, and poor scalability over time. LifeLearner addresses these challenges by enabling efficient on-device learning that adapts to new tasks without losing prior knowledge.

System Architecture and Methodology

LifeLearner introduces a three-pronged approach to make on-device learning viable:

  1. Rehearsal-based Meta-CL: Unlike previous meta-learning methods that rely solely on regularization, LifeLearner uses a Replay Buffer. It employs a "fast" inner loop to adapt to new classes quickly and a "slow" outer loop that uses replayed samples to refresh the model’s memory of old classes.
  2. Latent Replay & Algorithm Co-design: Instead of storing raw input data, which is bulky and computationally expensive, LifeLearner stores latent representations (activations) from an intermediate layer. By freezing the "feature extractor" part of the model during deployment, the system only needs to train the final classifier, drastically reducing the computational load.
  3. Two-Stage Compression Module: To fit the replay buffer into tiny memory spaces, LifeLearner uses Sparse Bitmap Compression (lossless) and Product Quantization (PQ) (lossy).
Figure 4: LifeLearner system architecture featuring a frozen feature extractor, a compression module for latent activations, and a continually learned classifier.

The researchers observed that latent activations in deep networks are incredibly sparse (often >90% zeros). By designing a compression module specifically to "filter" these zeros, they unlocked massive memory savings without hurting accuracy. Furthermore, by freezing the feature extractor on-device, they bypassed the need for full-model backpropagation, making the "rehearsal" of old tasks much faster and more energy-efficient.

Experimental Results

The system was evaluated on three diverse datasets: CIFAR-100, MiniImageNet, and GSCv2 (Google Speech Commands v2).

LifeLearner performed within 2.8% of an "Oracle" baseline (a model with access to all data at once) and outperformed existing Meta-CL methods by 4.1–16.1%. As shown in Figure 6, LifeLearner maintains competitive accuracy even as the number of learned classes increases, significantly outperforming standard meta-learning approaches like ANML and OML+AIM.

Figure 6: Accuracy comparison on the MiniImageNet dataset. LifeLearner tracks closely with the Oracle baseline and outperforms other Meta-CL methods as the number of classes increases.

Compared to the state-of-the-art ANML+AIM method, LifeLearner reduced the memory footprint by 178.7x and latency/energy consumption by 80.8–94.2%. The authors successfully deployed the system on an STM32H747 microcontroller with only 512 KB of SRAM—a platform 2,000x smaller than a Raspberry Pi 3B+.

Conclusion and Future Work

LifeLearner demonstrates that sophisticated meta-continual learning is possible on extremely resource-constrained hardware. While highly effective, future work could address potential performance drops during extreme domain shifts where the frozen feature extractor might become a bottleneck. Additionally, exploring stricter quantization levels (e.g., 1-4 bits) and automating layer selection for latent replay could further optimize the system for diverse edge applications.

Summary updated

BibTeX
@inproceedings{kwon2023lifelearner,
  author = {Young D. Kwon and Jagmohan Chauhan and Hong Jia and Stylianos I. Venieris and Cecilia Mascolo},
  title = {{LifeLearner: Hardware-Aware Meta Continual Learning System for Embedded Computing Platforms}},
  booktitle = {SenSys},
  year = {2023},
  doi = {10.1145/3625687.3625804},
}
Figure