Exploring Binary Classification Hidden within Partial Label Learning

Hengheng Luo, Yabin Zhang, Suyun Zhao, Hong Chen, Cuiping Li

Partial Label Learning (PLL) addresses scenarios where each training instance is associated with a set of candidate labels, only one of which is the ground truth. This ambiguity makes standard multiclass classification challenging, as noisy signals from false positives in the candidate set can degrade model performance. Traditional approaches typically attempt to model the full multiclass distribution directly. This paper proposes a paradigm shift: reframing PLL as a hidden binary classification problem—specifically, determining whether a model's prediction falls within the candidate set.

Methodology: The PLL-BC Framework

The authors introduce PLL-BC (Partial Label Learning via Binary Classification), which leverages the inherent binary structure of the label space (candidates vs. non-candidates).

1. The Binary Shift

Instead of a direct multiclass loss, the framework constructs a binary classifier $f$ from a multiclass model $g$. This classifier estimates the probability $P(y \in s|x)$, effectively asking if the predicted label is valid given the candidate set $s$. The authors provide theoretical guarantees, proving that minimizing this binary risk effectively bounds the true multiclass classification error.

Theoretical framework of PLL-BC showing the relationship between different risk approximations.

2. Logit Adjustment via Weight Normalization

To transition from binary signals to multiclass predictions, the authors employ a Weight Normalization Strategy (BC-WN). Unlike "Average-based" methods that treat all candidates equally or "Identification-based" methods (BC-Max) that aggressively pick the highest logit, BC-WN uses a softmax-weighted combination of candidate logits. A temperature parameter ($\lambda$) allows the model to dynamically balance trust in its current best guess against other candidates, facilitating a smoother disambiguation process.

Comparison of test accuracy across different strategies, highlighting the robustness of Weight Normalization (BC-WN).

Experimental Evaluation

The PLL-BC method was evaluated on synthetic benchmarks (MNIST, Fashion-MNIST, Kuzushiji) and real-world datasets (e.g., Lost, Birdsong, Soccer Player).

  • Synthetic Data: PLL-BC consistently achieved state-of-the-art (SOTA) results. For instance, on MNIST with a noise level of $q=0.2$, it reached 98.76% accuracy.
  • Real-World Data: The method showed significant gains, particularly on the Lost dataset (79.41%) and Soccer Player (57.43%), outperforming established methods like PRODEN and LWS.

Ablation studies confirmed that the Weight Normalization strategy is crucial for robustness. As shown in the accuracy curves, BC-WN avoids the early-training pitfalls of BC-Max, which can lock onto incorrect labels prematurely, especially on complex datasets like Kuzushiji where initial guesses are often noisy.

Conclusion

By identifying the binary classification task hidden within PLL, the authors provide a simplified yet theoretically grounded optimization landscape. The proposed logit adjustment strategy effectively manages label ambiguity, leading to superior performance across diverse benchmarks. Future work may extend this approach to handle noisy candidate sets where the ground truth might be missing, or explore its scalability with large-scale transformer architectures.

Summary updated

Abstract

Partial label learning (PLL) is to learn a discriminative model under incomplete supervision, where each instance is annotated with a candidate label set. The basic principle of PLL is that the unknown correct label y of an instance x resides in its candidate label set s, i.e., P(y ∈ s | x) = 1. On which basis, current researches either directly model P(x | y) under different data generation assumptions or propose various surrogate multiclass losses, which all aim to encourage the model-based Pθ(y ∈ s | x)→1 implicitly. In this work, instead, we explicitly construct a binary classification task toward P(y ∈ s | x) based on the discriminative model, that is to predict whether the model-output label of x is one of its candidate labels. We formulate a novel risk estimator with estimation error bound for the proposed PLL binary classification risk. By applying logit adjustment based on disambiguation strategy, the practical approach directly maximizes Pθ(y ∈ s | x) while implicitly disambiguating the correct one from candidate labels simultaneously. Thorough experiments validate that the proposed approach achieves competitive performance against the state-of-the-art PLL methods.
BibTeX
@inproceedings{luo2022exploring,
  author = {Hengheng Luo and Yabin Zhang and Suyun Zhao and Hong Chen and Cuiping Li},
  title = {{Exploring Binary Classification Hidden within Partial Label Learning}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.24963/ijcai.2022/456},
}
Figure