Particle Flow for Learning from Label Proportions

Alain Rakotomamonjy, Maxime Vono, Liva RalaivolaICML 2026 · OpenReview

TL;DR

This paper introduces FlowLLP, a novel method for learning from label proportions by framing it as a distribution alignment problem. It uses a particle flow approach to align "anchor" samples with known labels to unlabeled data in bags, achieving state-of-the-art performance on various datasets and outperforming existing methods.

What Problem Does This Paper Address?

Many real-world scenarios involve learning from data where individual labels are unavailable or costly to obtain. Instead, we might only have access to "bags" of data, each associated with aggregate label information, such as the proportion of each class present in the bag. This is known as Learning from Label Proportions (LLP). The challenge is to train an accurate instance-level classifier using only this coarse, bag-level supervision. Existing methods often struggle to effectively leverage this information, leading to suboptimal performance, especially in complex, multi-class scenarios.

Methods

FlowLLP tackles LLP by jointly optimizing three objectives:

  1. Bag-level loss: Measures the discrepancy between the true and predicted label proportions within each bag.
  2. Instance-level loss: Utilizes "anchor" samples with known labels and trainable supports. This is inspired by domain adaptation techniques.
  3. Distribution discrepancy: Aligns the learned support of the anchor samples with the distributions of the bag samples.

The core of the method is an alternating optimization process. The classifier is updated, and then the anchor distributions are aligned using a particle flow method. This flow is guided by a vector field designed to minimize the anchor loss while ensuring alignment between anchor and bag distributions. The paper also provides a theoretical analysis of this particle flow, guaranteeing convergence and identifying conditions for effective alignment, highlighting the importance of label proportion "gap" and "diversity" within bags.

Datasets

The authors evaluated FlowLLP on a variety of datasets, including:

  • Tabular datasets: Adult, Dry Beans, HAR, Indoor, Mushroom, Musk, Optdigits, Phoneme, Students, Telescope, Wine (from UCI repository and OpenML).
  • Image datasets: CIFAR-10, MNIST.

Key Insights and Ideas

  • LLP as Distribution Alignment: Reframing LLP as a distribution alignment problem, specifically using domain adaptation principles, is a powerful approach.
  • Anchor Particles as Proxies: Learnable "anchor" particles with known labels act as crucial proxies for fully labeled data, bridging the gap between bag-level and instance-level information.
  • Particle Flow for Alignment: The particle flow mechanism, guided by Wasserstein distance, provides an effective way to dynamically align anchor distributions with bag distributions.
  • Gap and Diversity are Crucial: Theoretical analysis and empirical results strongly suggest that a significant "gap" (difference in proportions) and "diversity" (variation in proportions across bags) in label proportions within bags are critical for successful learning.

Key Results

  • State-of-the-Art Performance: FlowLLP consistently outperforms existing LLP methods on both tabular and image datasets, achieving the best performance in 44 out of 54 multi-class experiments and showing strong results on binary datasets.
  • Robustness to Initialization: The method shows weak sensitivity to the initialization of anchor distributions, making it practical and easy to use.
  • End-to-End Learning: FlowLLP can be effectively adapted for end-to-end learning, leveraging pre-trained models (like ResNet) to achieve good performance on image datasets.
  • Theoretical Guarantees: The paper provides theoretical analysis guaranteeing the convergence of the particle flow and identifying conditions for effective alignment.

Limitations and Future Work

  • Sensitivity to Noisy Labels: FlowLLP is more sensitive to noisy label proportions than LLPGAN, especially at higher noise levels. This is attributed to the anchor flow mechanism directly relying on these proportions.
  • Computational Cost: While manageable for typical bag sizes, the optimal transport computation can be a bottleneck. The authors suggest exploring more efficient OT solvers.
  • Diversity vs. Gap: While diversity in label proportions is crucial for FlowLLP, a large gap alone is not sufficient. For scenarios lacking diversity, LLPGAN might be more robust.
  • Future Work: The authors suggest further investigation into optimizing the computational cost, exploring more robust methods for noisy label proportions, and extending the approach to other weakly supervised learning settings.

Summary updated

Abstract

This work proposes a novel method for solving learning from label proportion problems. For this purpose, we learn a classifier that minimizes three key objectives: (i) a bag-level loss, which quantifies the discrepancy between true and predicted label proportions in bags, (ii) an instance-level loss, inspired from domain adaptation, which leverages anchor samples with known labels and trainable supports and (iii) a distribution discrepancy that aims at aligning anchor's learned support with those of the bag samples. The problem is formulated as an alternating optimization process, iteratively updating the classifier and aligning distributions via a particle flow method. The flow of anchor samples is governed by a vector field designed to minimize the anchor loss while ensuring alignment between anchor and bag distributions. We provide a theoretical analysis, guaranteeing the convergence of the flow and identifying conditions under which the method achieves effective alignment. Our analysis highlights that gap and diversity in label proportions within bags is a critical factor for learnability. Empirical results on tabular and image datasets demonstrate the method's effectiveness, outperforming state-of-the-art approaches.
BibTeX
@inproceedings{rakotomamonjy2026particle,
  author = {Alain Rakotomamonjy and Maxime Vono and Liva Ralaivola},
  title = {{Particle Flow for Learning from Label Proportions}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=WkPYWGHQR2},
}
Figure