Decompositional Generation Process for Instance-Dependent Partial Label Learning

Congyu Qiao, Ning Xu, Xin Geng

Overview

IDGP (Instance-Dependent Partial Label Learning via Decompositional Generation Process) is a novel framework designed to tackle the realistic challenge of partial label learning (PLL) where candidate labels are biased by the data's features. By modeling label generation as a two-stage process—first picking the correct label, then adding confusing but related incorrect ones—IDGP leverages Maximum A Posteriori (MAP) estimation to achieve state-of-the-art accuracy on both synthetic and real-world datasets.

The Challenge of Instance-Dependent PLL

In Partial Label Learning (PLL), each training instance is assigned a set of candidate labels, only one of which is actually correct. For example, an image of a "Husky" might be labeled as {Dog, Wolf, Fox}.

Traditional PLL assumes that incorrect labels are added randomly (instance-independent). However, in the real world, annotators are more likely to confuse an object with something that looks visually or semantically similar. This is known as instance-dependent PLL. Previous methods often struggle in this scenario because they overfit to these biased candidate labels. Most existing methods fail to explicitly model how these confusing labels are generated, leading to sub-optimal performance when the label noise is structured and feature-related.

The IDGP Methodology

The core innovation of this paper is the Decompositional Generation Process. The authors assume the candidate label set is formed in two sequential stages:

  1. Correct Label Selection: The "true" label emerges in the annotator's mind, modeled by a Categorical distribution.
  2. Incorrect Label Selection: Related but incorrect labels are subsequently selected due to uncertainty, modeled by a Bernoulli distribution.

To optimize this process, the authors move beyond simple Maximum Likelihood (ML) and utilize Maximum A Posteriori (MAP) estimation. This allows them to incorporate "prior" information using:

  • Dirichlet distributions as priors for the Categorical model.
  • Beta distributions as priors for the Bernoulli model.

The "Refinement" Trick: Neural networks exhibit a "memorization effect," tending to learn simple, correct patterns before overfitting to noise. IDGP exploits this by using the model's own early-stage predictions to iteratively refine the priors for the Dirichlet and Beta distributions. This effectively "cleans" the label set as training progresses. Furthermore, by choosing conjugate priors (Dirichlet for Categorical, Beta for Bernoulli), the MAP estimation remains mathematically tractable and easy to implement within a standard deep learning backpropagation loop.

Key Results

The authors evaluated IDGP on two types of data: manually corrupted benchmarks (like CIFAR-10 and CIFAR-100) and real-world PLL datasets (like Yahoo!News and Lost). IDGP consistently outperformed eight state-of-the-art baselines, including PICO, VALEN, and PRODEN.

Benchmark Superiority

On manually corrupted datasets designed to simulate instance-dependent noise, IDGP demonstrated clear superiority. For instance, on CIFAR-10, IDGP achieved 87.57% accuracy, outperforming the next best method (PLCR) by over 1%. On the more complex CIFAR-100 dataset, it maintained a strong lead with 64.59%.

Benchmark Results

Real-World Performance

The framework's robustness extends to real-world datasets across diverse domains such as images, audio, and text. IDGP ranked first on 4 out of 5 real-world datasets tested. Notably, on the Yahoo!News dataset, it achieved 66.62% accuracy, significantly higher than traditional PLL methods.

Real-World Results

An ablation study comparing IDGP to "IDGP-ML" (a version without the refined MAP priors) showed a massive performance gap—for example, 77.02% vs 52.50% on the Lost dataset. This proves that the decompositional modeling and prior refinement are the critical components of the framework's success.

Limitations and Future Directions

While highly effective, IDGP has a few limitations:

  • Computational Overhead: The method requires an auxiliary branch to estimate parameters for the Bernoulli distribution, which increases the number of parameters and training time compared to simpler PLL methods.
  • Hyperparameter Sensitivity: Performance is somewhat sensitive to the scaling factors used for the Dirichlet and Beta distributions. While the authors provide suggested ranges, these may require tuning for entirely new domains.

Looking forward, the authors suggest that this decompositional approach could be extended to other weakly supervised tasks, such as learning with noisy labels or multi-label learning with missing data.

Summary updated

Abstract

Partial label learning (PLL) is a typical weakly supervised learning problem, where each training example is associated with a set of candidate labels among which only one is true. Most existing PLL approaches assume that the incorrect labels in each training example are randomly picked as the candidate labels and model the generation process of the candidate labels in a simple way. However, these approaches usually do not perform as well as expected due to the fact that the generation process of the candidate labels is always instance-dependent. Therefore, it deserves to be modeled in a refined way. In this paper, we consider instance-dependent PLL and assume that the generation process of the candidate labels could decompose into two sequential parts, where the correct label emerges first in the mind of the annotator but then the incorrect labels related to the feature are also selected with the correct label as candidate labels due to uncertainty of labeling. Motivated by this consideration, we propose a novel PLL method that performs Maximum A Posterior(MAP) based on an explicitly modeled generation process of candidate labels via decomposed probability distribution models. Extensive experiments on manually corrupted benchmark datasets and real-world datasets validate the effectiveness of the proposed method.
BibTeX
@inproceedings{qiao2023decompositional,
  author = {Congyu Qiao and Ning Xu and Xin Geng},
  title = {{Decompositional Generation Process for Instance-Dependent Partial Label Learning}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
  url = {https://openreview.net/forum?id=lKOfilXucGB},
}
Figure