Abstract
Partial label (PL) learning tackles the problem where each training instance is associated with a set of candidate labels that include both the true label and some irrelevant noise labels. In this paper, we propose a novel multi-level generative model for partial label learning (MGPLL), which tackles the PL problem by learning both a label level adversarial generator and a feature level adversarial generator under a bi-directional mapping framework between the label vectors and the data samples. MGPLL uses a conditional noise label generation network to model the non-random noise labels and perform label denoising, and uses a multi-class predictor to map the training instances to the denoised label vectors, while a conditional data feature generator is used to form an inverse mapping from the denoised label vectors to data samples. Both the noise label generator and the data feature generator are learned in an adversarial manner to match the observed candidate labels and data features respectively. We conduct extensive experiments on both synthesized and real-world partial label datasets. The proposed approach demonstrates the state-of-the-art performance for partial label learning.
Introduction
In Partial Label Learning (PLL), each training example is associated with a set of candidate labels, only one of which is correct. For example, a photo of a "computer" might be labeled as {computer, TV, monitor}. The challenge is to identify the true label hidden among the noise.
Traditional PLL research often assumes that noise labels are added to a dataset at random. However, in real-world scenarios, noise is often context-dependent and semantically related to the true label. For instance, a "computer" is more likely to be mislabeled as a "TV" than a "tree." By ignoring this non-random structure, existing models struggle with complex datasets. MGPLL addresses this by treating noise as a structured, conditional process, allowing the model to effectively "subtract" noise to find the ground truth.
Methods
The Multi-level Generative model for Partial Label Learning (MGPLL) utilizes a bi-directional mapping framework consisting of five neural networks:
- Conditional Noise Label Generator ($G_n$): Models noise generation based on the true label.
- Label Discriminator ($D_n$): Distinguishes between real candidate sets and those synthesized by $G_n$.
- Conditional Data Feature Generator ($G_x$): Learns the inverse mapping from labels to data features.
- Feature Discriminator ($D_x$): Distinguishes between real data samples and those generated by $G_x$.
- Prediction Network ($F$): A classifier mapping input features to denoised true labels.
The system is trained using a minimax adversarial game. While generators produce realistic noise and features, discriminators enforce accuracy. Simultaneously, the prediction network $F$ is trained to find the true label that satisfies both label-level and feature-level constraints. This bi-directional consistency ($x \to z$ and $z \to x$) provides strong regularization. To ensure training stability and avoid mode collapse, the authors employed the Wasserstein GAN (WGAN) framework.
Experiments
MGPLL was evaluated on both synthetic and real-world datasets:
- Synthetic Datasets: Six UCI benchmarks (ecoli, deter, vehicle, segment, satimage, letter) with injected random and non-random (target-dependent) noise.
- Real-World Datasets: Standard PLL benchmarks including FG-NET, Lost, Yahoo! News, MSRCv2, and BirdSong.
Results
MGPLL demonstrated state-of-the-art performance across various benchmarks.
Key findings include:
- Superiority in Non-Random Noise: MGPLL significantly outperformed methods like SURE and PALOC on datasets with target-dependent noise.
- Real-World Gains: Accuracy improved by 5.2% on MSRCv2 and 3.4% on Yahoo! News compared to the next best alternative.
- Ablation Study: Removing the label-level adversarial component led to a significant performance drop, confirming the importance of modeling the noise distribution.
Conclusion
MGPLL introduces a robust framework for PLL by explicitly modeling non-random label noise. While the model's complexity—requiring five networks—and the handling of discrete label spaces present challenges, its performance on real-world datasets is compelling. Future work includes extending this generative approach to Partial Multi-Label Learning (PML).
Summary updated