Candidate-aware Selective Disambiguation Based On Normalized Entropy for Instance-dependent Partial-label Learning

Shuo He, Guowu Yang, Lei Feng

Introduction: The Challenge of Instance-dependent Partial-label Learning

Partial-label learning (PLL) is a framework where each training instance is associated with a set of candidate labels, only one of which is the ground truth. Traditional PLL research often assumes that the "extra" incorrect labels are chosen at random (instance-independent). However, in real-world scenarios, labeling errors are rarely random; an annotator is far more likely to mistake a "Huskie" for a "Wolf" than for a "Whale."

This phenomenon is known as Instance-dependent PLL (ID-PLL). The primary challenge here is under-disambiguation (UD). Because incorrect candidate labels are often semantically similar to the true label, models struggle to distinguish between them, resulting in "flat" prediction probabilities across the candidate set. Standard PLL methods typically fail in these cases because they force the model to commit to a single label too early, leading to noisy supervision and degraded performance.

Comparison between random candidate labels and instance-dependent candidate labels, illustrating the under-disambiguation problem.

Methodology: Selective Disambiguation via Normalized Entropy

To address the UD problem, this paper proposes a two-stage framework centered on Normalized Entropy (NE). NE serves as a metric to gauge how effectively a model has disambiguated an instance. Unlike standard entropy, NE accounts for the varying number of candidate labels across different examples, providing a fairer basis for comparison.

Stage 1: Selective Training

The framework categorizes training examples into two groups based on their NE:

  1. Well-Disambiguated (WD) Examples: Instances with low NE, where the model shows high confidence in a specific candidate. These receive strong supervision.
  2. Under-Disambiguated (UD) Examples: Instances where the model remains uncertain. Instead of forcing a guess, the model utilizes complementary supervision, learning from labels known to be false (those outside the candidate set). This provides "harmless" information that aids convergence without introducing noise.
Disambiguation accuracy over time, showing the high performance of Well-Disambiguated (WD) sets compared to Under-Disambiguated (UD) sets.

Stage 2: Candidate-aware Thresholding (CAT)

Recognizing that some classes are inherently harder to learn than others, the authors implement Candidate-aware Thresholding (CAT). This involves class-wise WD-NE thresholds rather than a single global value, preventing easy-to-learn classes from dominating the training process. Additionally, a dual-network agreement strategy is employed: an example is only promoted to the WD set if both networks agree on the predicted label and the NE is sufficiently low.

Experimental Results

The proposed method was evaluated on several benchmarks, including CIFAR-10 and CIFAR-100, using simulated ID-PLL labels. The results demonstrate a significant performance leap over existing state-of-the-art (SOTA) methods:

  • CIFAR-10: The method achieved 95.04% accuracy, outperforming top competitors like ABLE (92.25%) and PiCO (91.15%).
  • CIFAR-100: It reached 78.67%, a nearly 4% improvement over the previous best, CRDPLL (77.82%).

The framework proved remarkably robust, maintaining stable performance even as the degree of label ambiguity ($q$) increased, a scenario where traditional PLL methods typically see sharp declines.

Conclusion

By using Normalized Entropy as a "progress bar" for learning, this framework effectively navigates the complexities of instance-dependent noise. The combination of selective disambiguation and class-wise thresholding allows the model to leverage high-confidence predictions while safely extracting information from ambiguous examples through complementary labels. While the dual-network approach adds some computational overhead, the substantial gains in accuracy and robustness make it a powerful solution for real-world partial-label learning tasks.

Summary updated

Abstract

In partial-label learning (PLL), each training example has a set of candidate labels, among which only one is the true label. Most existing PLL studies focus on the instance-independent (II) case, where the generation of candidate labels is only dependent on the true label. However, this II-PLL paradigm could be unrealistic, since candidate labels are usually generated according to the specific features of the instance. Therefore, instance-dependent PLL (ID-PLL) has attracted increasing attention recently. Unfortunately, existing ID-PLL studies lack an insightful perception of the intrinsic challenge in ID-PLL. In this paper, we start with an empirical study of the dynamics of label disambiguation in both II-PLL and ID-PLL. We found that the performance degradation of ID-PLL stems from the inaccurate supervision caused by massive under-disambiguated (UD) examples that do not achieve complete disambiguation. To solve this problem, we propose a novel two-stage PLL framework including selective disambiguation and candidate-aware thresholding. Specifically, we first choose a part of well-disambiguated (WD) examples based on the magnitude of normalized entropy (NE) and integrate harmless complementary supervision from the remaining ones to train two networks. Next, the remaining examples whose NE is lower than the specific class-wise WD-NE threshold are selected as additional WD ones. Meanwhile, the remaining UD examples, whose NE is lower than the self-adaptive UD-NE threshold and whose predictions from two networks are agreed, are also regarded as WD ones for model training. Extensive experiments demonstrate that our proposed method outperforms state-of-the-art PLL methods.
BibTeX
@inproceedings{he2023candidate,
  author = {Shuo He and Guowu Yang and Lei Feng},
  title = {{Candidate-aware Selective Disambiguation Based On Normalized Entropy for Instance-dependent Partial-label Learning}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2023},
  doi = {10.1109/iccv51070.2023.00172},
}
Figure