Decomposition-based Generation Process for Instance-Dependent Partial Label Learning

Congyu Qiao, Ning Xu, Xin Geng

TL;DR

This paper tackles the challenge of "instance-dependent partial label learning" (IDPLL) by proposing a novel method, IDGP, that explicitly models how incorrect labels are generated alongside the true label for each data point. By decomposing this generation process and leveraging a Maximum A Posteriori (MAP) approach with refined prior information, IDGP significantly improves classification accuracy on various datasets compared to existing methods.

What Problem Does This Paper Address?

Partial Label Learning (PLL) is a common scenario where each data point comes with a set of possible labels, but only one is correct. Traditional PLL methods often assume that the incorrect labels are chosen randomly and independently of the data instance itself. However, in real-world applications, the generation of these candidate labels is often instance-dependent – meaning the features of the data point influence which incorrect labels are mistakenly included. This paper addresses the limitations of existing methods by explicitly modeling this instance-dependent label generation process, which is crucial for achieving better performance.

Methods

The core innovation of this paper is the Decompositional Generation Process for instance-dependent partial labels. The authors propose that the generation of candidate labels can be broken down into two stages:

  1. Correct Label Generation: The true label for an instance is first considered. This is modeled using a Categorical distribution, parameterized by instance-specific probabilities.
  2. Incorrect Label Generation: Then, incorrect labels related to the instance's features are selected. This is modeled using a multivariate Bernoulli distribution, also parameterized by instance-specific probabilities.

These two distributions are then integrated into a Maximum A Posteriori (MAP) framework. To facilitate learning and leverage prior knowledge, the authors employ Dirichlet distributions as priors for the Categorical parameters and Beta distributions for the Bernoulli parameters. The method iteratively refines these prior distributions throughout training, effectively "learning" the label generation process itself. The proposed method is named IDGP (Instance-dependent partial label learning via Decompositional Generation Process).

Datasets

The authors evaluate IDGP on several widely used benchmark datasets:

  • MNIST
  • Kuzushiji-MNIST
  • Fashion-MNIST
  • CIFAR-10
  • CIFAR-100

Additionally, they test IDGP on five real-world datasets:

  • Lost
  • BirdSong
  • MSRCv2
  • Soccer Player
  • Yahoo!News

For the benchmark datasets, instance-dependent partial labels were generated using the same strategy as in Xu et al. (2021). For the real-world datasets, a 80%/10%/10% train/validation/test split was used.

Key Insights and Ideas

  • Instance-Dependent Label Generation is Key: The paper strongly argues that assuming random, instance-independent label generation is a significant oversimplification that hinders performance in real-world PLL scenarios.
  • Decompositional Approach: Breaking down the label generation into "correct" and "incorrect" stages, each with its own probabilistic model, provides a more nuanced and accurate representation.
  • Iterative Prior Refinement: The use of MAP estimation with conjugate priors (Dirichlet and Beta) and their iterative refinement allows the model to learn the underlying label generation process and leverage this information to improve classification. This is a crucial aspect that distinguishes IDGP from methods that only focus on the classification task.
  • Explicit Modeling of Uncertainty: By modeling the generation process, IDGP implicitly captures and leverages the uncertainty inherent in partial label assignments.

Key Results

IDGP consistently outperforms several state-of-the-art DNN-based PLL algorithms across both benchmark and real-world datasets.

  • On benchmark datasets, IDGP achieved the best results, with improvements particularly noticeable on Fashion-MNIST and CIFAR-10.
  • On real-world datasets, IDGP also demonstrated strong performance, often outperforming other methods, with comparable results to some top performers on specific datasets.
  • An ablation study showed that removing the prior refinement mechanism (IDGP-ML) significantly degraded performance, highlighting the importance of the learned generation process.
  • Parameter sensitivity analysis indicated that IDGP is relatively stable, with suggested parameter ranges varying slightly for small-scale vs. large-scale datasets.

Limitations and Future Work

  • Computational Cost: While not explicitly detailed, modeling the generation process and iterative refinement likely increases computational complexity compared to simpler PLL methods.
  • Data Augmentation: The paper notes that PLCR and PICO baselines could not be compared on real-world datasets due to data augmentation requirements, suggesting that data augmentation strategies might need to be adapted for certain types of real-world data in PLL.
  • Future Work: The paper's success in explicitly modeling instance-dependent label generation opens doors for further research into more sophisticated generative models for partial labels and exploring its application in other weakly supervised learning settings. The authors also suggest that the proposed method can be extended to accommodate uniform generation processes (Appendix A.2).

Summary updated

Abstract

Partial label learning (PLL) is a typical weakly supervised learning problem, where each training example is associated with a set of candidate labels among which only one is true. Most existing PLL approaches assume that the incorrect labels in each training example are randomly picked as the candidate labels and model the generation process of the candidate labels in a simple way. However, these approaches usually do not perform as well as expected due to the fact that the generation process of the candidate labels is always instance-dependent. Therefore, it deserves to be modeled in a refined way. In this paper, we consider instance-dependent PLL and assume that the generation process of the candidate labels could decompose into two sequential parts, where the correct label emerges first in the mind of the annotator but then the incorrect labels related to the feature are also selected with the correct label as candidate labels due to uncertainty of labeling. Motivated by this consideration, we propose a novel PLL method that performs Maximum A Posterior (MAP) based on an explicitly modeled generation process of candidate labels via decomposed probability distribution models. Extensive experiments on manually corrupted benchmark datasets and real-world datasets validate the effectiveness of the proposed method. Source code is available at https://github.com/palm-ml/idgp.
BibTeX
@inproceedings{qiao2023decomposition,
  author = {Congyu Qiao and Ning Xu and Xin Geng},
  title = {{Decomposition-based Generation Process for Instance-Dependent Partial Label Learning}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
  url = {https://openreview.net/forum?id=lKOfilXucGB},
}
Figure