Few Sample Learning without Data Storage for Lifelong Stream Mining (Student Abstract)

Zhuoyi Wang, Yigong Wang, Yu Lin, Bo Dong, Hemeng Tao, Latifur KhanAAAI 2020 · OpenReview

Abstract

Continuously mining complexity data stream has recently been attracting an increasing amount of attention, due to the rapid growth of real-world vision/signal applications such as self-driving cars and online social media messages. In this paper, we aim to address two significant problems in the lifelong/incremental stream mining scenario: first, how to make the learning algorithms generalize to the unseen classes only from a few labeled samples; second, is it possible to avoid storing instances from previously seen classes to solve the catastrophic forgetting problem? We introduce a novelty stream mining framework to classify the infinite stream of data with different categories that occurred during different times. We apply a few-sample learning strategy to make the model recognize the novel class with limited samples; at the same time, we implement an incremental generative model to maintain old knowledge when learning new coming categories, and also avoid the violation of data privacy and memory restrictions simultaneously. We evaluate our approach in the continual class-incremental setup on the classification tasks and ensure the sufficient model capacity to accommodate for learning the new incoming categories.

Introduction

In the domain of "stream mining," data arrives as an infinite, ever-changing flow. Traditional Deep Neural Networks (DNNs) face two primary hurdles in this environment: Data Scarcity and Catastrophic Forgetting. Learning a new category typically requires thousands of human-annotated samples, which is often impractical. Furthermore, when a model learns new information, it tends to overwrite previous knowledge. While many systems mitigate this by storing a subset of old data for "rehearsal," this approach poses significant challenges regarding memory limits and data privacy—especially in sensitive applications like autonomous driving or personal social media.

This research proposes a framework that learns from a "few-shot" set of new samples while maintaining historical knowledge without storing any actual instances from the past.

Methodology

The proposed framework integrates two primary modules to balance plasticity and stability:

1. Few-Sample Learning Module

To handle new classes with minimal data, the framework utilizes a meta-learning paradigm. Instead of training a classifier from scratch for every new class, a fixed feature extractor ($f_\theta$) is used to generate "prototypes" (representative vectors) for each category. Classification is then performed based on the distance between new inputs and these prototypes. By adopting a strategy similar to Matching Networks, the model learns how to learn quickly, requiring only a new weight matrix ($W_N$) for novel classes.

2. Incremental Generative Model

To solve forgetting without data storage, the authors implement an Incremental Generative Model based on a Conditional GAN.

  • Generative Replay: Instead of saving historical images, the model learns to generate synthetic versions of old data. During new tasks, the model "replays" these synthetic images to reinforce past knowledge.
  • Weight Masking: To prevent the generator from losing its ability to recreate old data, binary masks are applied to the network weights. These masks protect the parameters used for previous tasks, ensuring updates for new knowledge only occur in "free" portions of the network.

Experimental Results

The approach was evaluated on two standard benchmarks: CUB200 (fine-grained bird species) and CIFAR10 (general object categories). The researchers simulated a stream by splitting these datasets into base and novel classes.

Performance comparison of the proposed framework against baselines on CUB and CIFAR10 datasets.

As shown in the results table, the proposed framework ("Ours") consistently outperforms baselines including SAND, standard DNN fine-tuning, and P-Net (Matching Networks). Notably, on the CUB200 5-sample task, the model achieved 79.17% accuracy, a significant improvement over P-Net's 70.17%. Even in the extreme 1-sample scenario, the model maintained a strong lead, reaching 60.23% on CUB200.

Discussion and Conclusion

The shift from "data storage" to "model-based memory" represents a significant advancement for privacy-preserving AI. By using a GAN to recreate the past and weight masking to manage network plasticity, the system effectively addresses the stability-plasticity dilemma.

Future work may address the computational overhead of training incremental GANs and the long-term scalability of the masking strategy. As the number of tasks grows, the availability of "free" weights may become a bottleneck, necessitating further research into dynamic network expansion or more efficient parameter isolation techniques.

Summary updated

Abstract

Continuously mining complexity data stream has recently been attracting an increasing amount of attention, due to the rapid growth of real-world vision/signal applications such as self-driving cars and online social media messages. In this paper, we aim to address two significant problems in the lifelong/incremental stream mining scenario: first, how to make the learning algorithms generalize to the unseen classes only from a few labeled samples; second, is it possible to avoid storing instances from previously seen classes to solve the catastrophic forgetting problem? We introduce a novelty stream mining framework to classify the infinite stream of data with different categories that occurred during different times. We apply a few-sample learning strategy to make the model recognize the novel class with limited samples; at the same time, we implement an incremental generative model to maintain old knowledge when learning new coming categories, and also avoid the violation of data privacy and memory restrictions simultaneously. We evaluate our approach in the continual class-incremental setup on the classification tasks and ensure the sufficient model capacity to accommodate for learning the new incoming categories.
BibTeX
@inproceedings{wang2020few,
  author = {Zhuoyi Wang and Yigong Wang and Yu Lin and Bo Dong and Hemeng Tao and Latifur Khan},
  title = {{Few Sample Learning without Data Storage for Lifelong Stream Mining (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  url = {https://openreview.net/forum?id=fjviFbPIc-},
}
Figure