Uncertainty-Aware Hierarchical Refinement for Incremental Implicitly-Refined Classification

Jian Yang, Kai Zhu, Kecheng Zheng, Yang CaoNeurIPS 2022 · OpenReview

Incremental Implicitly-Refined Classification (IIRC) is a challenging paradigm in machine learning where a model must learn increasingly specific labels for objects over time. For instance, a model might first learn the broad category "Garment" and later need to recognize the more specific "Miniskirt." Most Class-Incremental Learning (CIL) methods assume new classes are entirely distinct from previous ones. When applied to IIRC, these standard methods often treat a "Miniskirt" as completely unrelated to a "Garment," causing the model to forget the hierarchical relationship.

This leads to two major issues: semantic drift (where subclasses drift away from their parent superclasses in the feature space) and feature confusion (where new classes without a parent superclass get mixed up with existing ones).

The IIRC setting compared to standard incremental learning.

To address this, researchers have proposed Uncertainty-Aware Hierarchical Refinement (UAHR), a novel framework designed to prevent the model from forgetting general categories while mastering specific new ones.

The UAHR Framework

The UAHR scheme introduces two primary strategies to maintain hierarchical integrity during incremental learning:

  1. Global Representation Extension (GRE): To reduce feature confusion, GRE uses a Radial Basis Function (RBF) kernel to measure the distance between class centers in the embedding space. During training, it explicitly optimizes a loss function to widen these distances. This ensures that different classes remain discriminative and do not overlap excessively.
  2. Hierarchical Distribution Alignment (HDA): This is the core "uncertainty-aware" mechanism. When new classes arrive, the model calculates their output entropy using the old model. If a new class consistently triggers high confidence in an old class, the system identifies an inheritance relationship. The model's own confusion effectively acts as a map to discover hidden hierarchies without needing a predefined taxonomy.
  3. Refined Distillation: Once relationships are identified, the model uses a selective distillation mechanism. It adjusts the optimization direction so that shifting subclasses are "pulled back" into the distribution of their parent superclass, ensuring consistency across the hierarchy and counteracting the natural tendency of new subclasses to "outgroup" their parents.

Experimental Validation

The researchers evaluated UAHR on four benchmarks specifically adapted for the IIRC task: IIRC-CIFAR, IIRC-ImageNet-lite, IIRC-ImageNet-Subset, and IIRC-ImageNet-full.

UAHR consistently outperformed state-of-the-art methods like Podnet, LUCIR, and iCaRL across all benchmarks. Quantitatively, on IIRC-CIFAR, UAHR achieved a final precision-weighted Jaccard similarity (PW-JS) of 19.05%, compared to roughly 15-17% for other top-performing methods.

Qualitatively, t-SNE visualizations demonstrate that UAHR produces much tighter, more distinct clusters for new classes. It successfully keeps subclasses within the "orbit" of their superclasses, whereas baseline methods show significant overlap and semantic drift.

t-SNE visualizations showing that UAHR creates clearer class separation and better subclass-superclass alignment than baselines.

Limitations and Future Directions

While UAHR demonstrates strong results, a few areas remain for exploration:

  • Computational Overhead: Calculating entropy distributions and RBF mappings for all class centers adds complexity to the training phase.
  • Exemplar Dependence: Like many CIL methods, UAHR relies on storing a small subset of old data (exemplars). Future work could investigate "exemplar-free" versions to better address data privacy concerns.
  • Complex Hierarchies: The current work focuses on a two-level (superclass-subclass) hierarchy. Real-world data often features deeper, more complex trees that may require further architectural adjustments.

Summary updated

Abstract

Incremental implicitly-refined classification task aims at assigning hierarchical labels to each sample encountered at different phases. Existing methods tend to fail in generating hierarchy-invariant descriptors when the novel classes are inherited from the old ones. To address the issue, this paper, which explores the inheritance relations in the process of multi-level semantic increment, proposes an Uncertainty-Aware Hierarchical Refinement (UAHR) scheme. Specifically, our proposed scheme consists of a global representation extension strategy that enhances the discrimination of incremental representation by widening the corresponding margin distance, and a hierarchical distribution alignment strategy that refines the distillation process by explicitly determining the inheritance relationship of the incremental class. Particularly, the shifting subclasses are corrected under the guidance of hierarchical uncertainty, ensuring the consistency of the homogeneous features. Extensive experiments on widely used benchmarks (i.e., IIRC-CIFAR, IIRC-ImageNet-lite, IIRC-ImageNet-Subset, and IIRC-ImageNet-full) demonstrate the superiority of our proposed method over the state-of-the-art approaches.
BibTeX
@inproceedings{yang2022uncertainty,
  author = {Jian Yang and Kai Zhu and Kecheng Zheng and Yang Cao},
  title = {{Uncertainty-Aware Hierarchical Refinement for Incremental Implicitly-Refined Classification}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=syLtcS9F32d},
}
Figure