TL;DR
This paper tackles the challenge of "instance-dependent partial label learning" (IDPLL) by proposing a novel method, IDGP, that explicitly models how incorrect labels are generated alongside the true label for each data point. By decomposing this generation process and leveraging a Maximum A Posteriori (MAP) approach with refined prior information, IDGP significantly improves classification accuracy on various datasets compared to existing methods.
What Problem Does This Paper Address?
Partial Label Learning (PLL) is a common scenario where each data point comes with a set of possible labels, but only one is correct. Traditional PLL methods often assume that the incorrect labels are chosen randomly and independently of the data instance itself. However, in real-world applications, the generation of these candidate labels is often instance-dependent – meaning the features of the data point influence which incorrect labels are mistakenly included. This paper addresses the limitations of existing methods by explicitly modeling this instance-dependent label generation process, which is crucial for achieving better performance.
Methods
The core innovation of this paper is the Decompositional Generation Process for instance-dependent partial labels. The authors propose that the generation of candidate labels can be broken down into two stages:
- Correct Label Generation: The true label for an instance is first considered. This is modeled using a Categorical distribution, parameterized by instance-specific probabilities.
- Incorrect Label Generation: Then, incorrect labels related to the instance's features are selected. This is modeled using a multivariate Bernoulli distribution, also parameterized by instance-specific probabilities.
These two distributions are then integrated into a Maximum A Posteriori (MAP) framework. To facilitate learning and leverage prior knowledge, the authors employ Dirichlet distributions as priors for the Categorical parameters and Beta distributions for the Bernoulli parameters. The method iteratively refines these prior distributions throughout training, effectively "learning" the label generation process itself. The proposed method is named IDGP (Instance-dependent partial label learning via Decompositional Generation Process).
Datasets
The authors evaluate IDGP on several widely used benchmark datasets:
- MNIST
- Kuzushiji-MNIST
- Fashion-MNIST
- CIFAR-10
- CIFAR-100
Additionally, they test IDGP on five real-world datasets:
- Lost
- BirdSong
- MSRCv2
- Soccer Player
- Yahoo!News
For the benchmark datasets, instance-dependent partial labels were generated using the same strategy as in Xu et al. (2021). For the real-world datasets, a 80%/10%/10% train/validation/test split was used.
Key Insights and Ideas
- Instance-Dependent Label Generation is Key: The paper strongly argues that assuming random, instance-independent label generation is a significant oversimplification that hinders performance in real-world PLL scenarios.
- Decompositional Approach: Breaking down the label generation into "correct" and "incorrect" stages, each with its own probabilistic model, provides a more nuanced and accurate representation.
- Iterative Prior Refinement: The use of MAP estimation with conjugate priors (Dirichlet and Beta) and their iterative refinement allows the model to learn the underlying label generation process and leverage this information to improve classification. This is a crucial aspect that distinguishes IDGP from methods that only focus on the classification task.
- Explicit Modeling of Uncertainty: By modeling the generation process, IDGP implicitly captures and leverages the uncertainty inherent in partial label assignments.
Key Results
IDGP consistently outperforms several state-of-the-art DNN-based PLL algorithms across both benchmark and real-world datasets.
- On benchmark datasets, IDGP achieved the best results, with improvements particularly noticeable on Fashion-MNIST and CIFAR-10.
- On real-world datasets, IDGP also demonstrated strong performance, often outperforming other methods, with comparable results to some top performers on specific datasets.
- An ablation study showed that removing the prior refinement mechanism (IDGP-ML) significantly degraded performance, highlighting the importance of the learned generation process.
- Parameter sensitivity analysis indicated that IDGP is relatively stable, with suggested parameter ranges varying slightly for small-scale vs. large-scale datasets.
Limitations and Future Work
- Computational Cost: While not explicitly detailed, modeling the generation process and iterative refinement likely increases computational complexity compared to simpler PLL methods.
- Data Augmentation: The paper notes that PLCR and PICO baselines could not be compared on real-world datasets due to data augmentation requirements, suggesting that data augmentation strategies might need to be adapted for certain types of real-world data in PLL.
- Future Work: The paper's success in explicitly modeling instance-dependent label generation opens doors for further research into more sophisticated generative models for partial labels and exploring its application in other weakly supervised learning settings. The authors also suggest that the proposed method can be extended to accommodate uniform generation processes (Appendix A.2).
Summary updated