Abstract
In dynamic machine learning environments, models must frequently adapt by learning new classes (Class Incremental Learning, CIL) and removing obsolete or sensitive data (Machine Unlearning, MU). Traditional approaches to these tasks often involve computationally expensive retraining or complex model-intrinsic updates that can lead to catastrophic forgetting or performance degradation. This paper introduces eCIL-MU, a novel framework that shifts the focus from modifying model weights to managing data within embedding-based vector databases. By utilizing a pre-trained embedding model and a dual-database system, eCIL-MU achieves "non-destructive" unlearning with massive efficiency gains. Experimental results on CIFAR-10 and CIFAR-100 benchmarks demonstrate speedups of up to 278× compared to traditional retraining methods while maintaining high accuracy on remaining data.
Introduction
Modern machine learning systems face a dual challenge: they must evolve sequentially to incorporate new knowledge while also possessing the ability to "forget" specific data to comply with privacy regulations like GDPR or to remove obsolete information. Class Incremental Learning (CIL) typically struggles with catastrophic forgetting, where new tasks overwrite previous knowledge. Conversely, Machine Unlearning (MU) is often treated as a separate, costly process involving retraining from scratch or specialized weight updates.
The authors identify a critical gap in existing literature: the need for a unified framework that handles both CIL and MU rapidly without compromising the underlying model's integrity. eCIL-MU addresses this by treating unlearning as a data management problem rather than a model optimization problem.
Methodology
The core of the eCIL-MU framework is a data-centric approach centered on high-dimensional embeddings. The architecture consists of several key components:
- Embedding Model ($M_e$): A pre-trained ResNet-50 is used to map input data into latent vectors, providing a stable representation space.
- Dual Vector Databases: The system maintains DB-CIL for active classes and DB-MU for classes that have been unlearned.
- The Unlearning Process: Instead of updating model weights, unlearning is performed by identifying relevant vectors in DB-CIL using K-Nearest Neighbor (KNN) and cosine similarity, then migrating them to DB-MU. This process is nearly instantaneous and fully reversible.
- Inference Filtering: During the inference phase, input data is embedded and checked against DB-MU. If the similarity exceeds a threshold ($s=0.77$), the sample is identified as "unlearned" and filtered.
To handle misidentified unlearned samples, the authors implemented a "Shift to nearest class" strategy. This reclassifies filtered samples to the closest valid class in the active database, preserving the model's overall discriminative ability even when the filter makes a mistake.
Key Results and Insights
The framework was evaluated using CIFAR-10 and CIFAR-100 datasets to simulate sequences of adding and removing classes.
Efficiency and Speedup
The most striking result is the computational efficiency. Because unlearning involves vector migration rather than weight updates, it can occur asynchronously with CIL training. In scenarios focused exclusively on unlearning, eCIL-MU achieved an acceleration of up to 278.48× compared to retraining from scratch.
Performance Preservation
Unlike model-intrinsic unlearning methods that often degrade performance on remaining classes ($C_r$), eCIL-MU maintains accuracy levels within ~2% of the original CIL model. On CIFAR-10, the vector migration process achieved a filtering accuracy of 87%, demonstrating that embedding-based matching is a precise mechanism for class removal.
Discussion and Limitations
While eCIL-MU offers significant advantages in speed and non-destructiveness, it introduces a dependency on the quality of the pre-trained embedding model. If the initial embeddings are not sufficiently representative of the domain, filtering accuracy may suffer. Additionally, the framework requires storage for the vector databases, which is a trade-off for the reduced computational load. Future work may explore optimizing these databases for larger-scale datasets and refining the similarity threshold selection process.
Summary updated