When a machine learning model learns new categories sequentially, the primary threat to its memory isn't just the passive fading of old weights. In dense or multi-label prediction tasks—such as semantic segmentation, object detection, and named entity recognition—the threat is often an active contradiction embedded within the new training data itself.
This problem stems from a phenomenon known as background shift or semantic shift. Incremental pseudo-labeling offers an elegant, data-free solution to this challenge by turning a destructive training signal into a constructive one.
The Hidden Trap of New Data
Consider a model trained to segment cars (Task 1). Later, we want to teach it to segment pedestrians (Task 2). The dataset for Task 2 explicitly labels pedestrians, but it does not annotate cars. Crucially, any cars appearing in the Task 2 images are lumped into a catch-all "background" or "void" class.
Standard Incremental Learning Step:
New Image: [ Car (Unlabeled) ] [ Pedestrian (Labeled) ]
Ground Truth: [ Background ] [ Pedestrian ]
▲
└── The model is penalized for recognizing the car!
When the model processes a Task 2 image containing both a pedestrian and a car, it correctly identifies the car based on its prior knowledge. However, the ground-truth label insists that those pixels are "background." The loss function penalizes the model for its correct prediction, forcing it to actively unlearn how to recognize cars.
This issue spans multiple domains:
- Semantic Segmentation: Old classes are re-assigned to the background pixel class.
- Object Detection: Unlabeled legacy objects are treated as negative background samples.
- Named Entity Recognition (NER): Previously learned entities are treated as the non-entity "O" class.
- Multi-Label Classification: Images containing multiple objects are only partially annotated for the current task's classes.
Turning Background into Supervision
The core intuition of incremental pseudo-labeling is simple: trust the model's past self to correct the current data's omissions.
Before updating the model's weights on a new task, a frozen copy of the model from the previous iteration is used as a teacher. This frozen model generates predictions on the new dataset. If the old model detects a previously learned class within the unannotated background regions, that prediction is converted into a pseudo-label.
Incremental Pseudo-Labeling Step:
New Image: [ Car (Unlabeled) ] [ Pedestrian (Labeled) ]
Old Model Pred: [ Car (High Conf) ] [ Unknown/Low Conf ]
│
▼ (Inpaint / Propagate)
Pseudo-Ground: [ Car ] [ Pedestrian ]
▲
└── The model now safely reinforces old knowledge.
By inpainting these pseudo-labels into the new task's ground truth, the ambiguous background signal is cleaned. As Chao Shang et al. demonstrate in Incrementer: Transformer for Class-Incremental Semantic Segmentation with Knowledge Distillation Focusing on Old Class, this process converts what would have been destructive noise into clear supervision, ensuring the model does not overwrite its historical knowledge.
Crucially, this strategy operates without requiring access to the original training data, making it highly effective for privacy-sensitive or memory-constrained environments where storing physical exemplars is prohibited.
Separating Signal from Noise
A major challenge of this approach is that the old model is not perfect; blindly accepting all of its predictions would introduce severe label noise into the training loop. To ensure that only reliable historical knowledge is preserved, researchers employ various filtering and calibration strategies:
Confidence and Entropy Thresholding
The simplest way to filter pseudo-labels is to enforce strict confidence boundaries. For instance, Zihan Lin et al. in Continual Semantic Segmentation via Structure Preserving and Projected Feature Alignment utilize class-specific entropy thresholds. By only converting background pixels into pseudo-labels when the old model's prediction entropy falls below a certain threshold, they ensure that the system only learns from its most certain historical representations.
Dynamic Thresholding
In complex environments like multi-label class-incremental learning, static thresholds can fail as the label space expands. To combat this, SongLin Dong et al. introduce a dynamic pseudo-labeling mechanism in Knowledge Restore and Transfer for Multi-Label Class-Incremental Learning. Their approach dynamically adjusts the acceptance thresholds based on the model's evolving state, preventing the model from treating known objects as background noise even when they appear in entirely new contexts.
Uncertainty-Aware Propagation
In spatial domains like 3D point clouds, geometric structure can be leveraged alongside model confidence. In Geometry and Uncertainty-Aware 3D Point Cloud Class-Incremental Semantic Segmentation, the authors estimate predictive uncertainty through stochastic sampling. They then filter out highly uncertain predictions and propagate high-confidence labels from spatial neighbors to clean up the semantic shift across sparse 3D points.
Universality Across Modalities
While heavily utilized in computer vision, the intuition behind background pseudo-labeling applies anywhere a catch-all "negative" class exists.
In 3D object detection, Na Zhao et al. apply this in Static-Dynamic Co-teaching for Class-Incremental 3D Object Detection using a dual-teacher strategy. A static teacher preserves old knowledge by generating bounding box pseudo-labels for historical categories, preventing the student detector from treating legacy objects as empty background space.
The same mechanism resolves the "Unlabeled Entity Problem" in natural language processing. In class-incremental NER, tokens belonging to old entity types are frequently misclassified as the non-entity "O" class. To solve this, Ruotian Ma et al. show in Learning "O" Helps for Learning More: Handling the Unlabeled Entity Problem for Class-incremental NER that reassessing background tokens based on their proximity to saved class prototypes allows them to successfully relabel tokens, providing the necessary pseudo-supervision to keep old entity definitions intact.