ProMeCD: Unifying Long-Tailed and Noisy Label Learning via White-Box Control

Yu ZhuICML 2026 · OpenReview

Deep learning models are notoriously data-hungry, thriving on massive, perfectly curated datasets. However, real-world data is inherently defective, frequently plagued by long-tailed distributions (extreme class imbalance) and label noise (incorrect annotations). Traditional solutions often rely on black-box meta-learning to re-weight samples during training. Yet, this approach introduces a fatal circular dependency: it requires a pristine, balanced validation set to guide the meta-learner. In practice, if such a clean dataset existed, complex meta-learning would be largely unnecessary. Furthermore, standard optimizers struggle with "scalar ambiguity," unable to distinguish between a rare "tail" sample and a "noisy" sample, as both yield high loss values.

To break this dependency, researchers introduce ProMeCD (Probabilistic Meta-learning of Cognitive Dynamics). ProMeCD fundamentally recasts optimization as an autonomous control problem rather than a bi-level neural network training task, replacing opaque meta-learners with a transparent Proportional-Integral (PI) controller.

The Mechanics of Cognitive Entropy

The core innovation of ProMeCD lies in how it perceives learning dynamics. Instead of relying solely on scalar loss values, ProMeCD models the direction of gradients using von Mises-Fisher (vMF) distributions.

From this, the system monitors Cognitive Entropy, a novel metric that combines magnitude uncertainty (signal strength) and directional uncertainty (signal consistency). This metric is the key to resolving scalar ambiguity. The researchers identified a crucial insight: while tail samples and noisy samples both exhibit high loss, their gradient structures are polar opposites. Tail samples demonstrate directional consistency (low entropy), whereas noisy samples are chaotic (high entropy).

A Decoupled, White-Box PI Controller

Leveraging cognitive entropy, ProMeCD employs a decoupled control strategy to manage the learning process without needing an external validation set:

  • Integral (I) Loop for Tail Classes: This loop accumulates "learning lag." If a class consistently points in the right direction but learns slowly (characteristic of minority classes), the controller boosts its weight.
  • Proportional (P) Loop for Noisy Labels: This loop penalizes "directional chaos." If a sample's gradients scatter unpredictably, the controller identifies it as noise and suppresses its influence.

Because ProMeCD relies entirely on internal gradient dynamics, it is a fully white-box, validation-free framework. The authors mathematically prove that this mechanism prevents the "Minority Initial Drop"—a phenomenon where rare classes are forgotten early in training—and ensures monotonic loss reduction.

Experimental Validation and Efficiency

The framework was rigorously evaluated across Long-Tailed Learning (CIFAR-10/100-LT, iNaturalist 2018), Noisy Label Learning (CIFAR-10/100-N, mini WebVision, Clothing1M), and unified scenarios containing both defects.

  • Superior Accuracy: ProMeCD significantly outperformed the recent meta-learner FMW-Net by over 10% in severe imbalance settings. For instance, on CIFAR-100-LT with an imbalance factor of 100, it achieved 53.12% accuracy compared to FMW-Net's 42.32%.
  • Robustness to Noise: Under a severe 40% "Flip" noise corruption, ProMeCD maintained a remarkable 91.12% accuracy on CIFAR-10-N, succeeding where traditional meta-learning frameworks faltered.
  • Computational Efficiency: Typical meta-learning approaches (like MW-Net) can add 27% to over 100% in training time overhead due to bi-level optimization. In contrast, ProMeCD's control-theory approach adds a mere 4.8% computational overhead compared to standard training.

Limitations and Future Directions

Despite its robust performance, ProMeCD has a few limitations. The authors initially explored a full PID (Proportional-Integral-Derivative) controller but found the Derivative (D) term too sensitive to the inherent stochastic noise of SGD, leading them to adopt the PI architecture.

Additionally, tracking full gradient states is memory-intensive for massive models. The authors mitigated this by using a "classifier-only" proxy for large datasets like iNaturalist, but optimizing full-state tracking for massive architectures remains an open challenge. Finally, while the PI gains ($\beta_p, \beta_i$) proved highly transferable across datasets, the system still relies on manual hyperparameter tuning for these settings.

Ultimately, ProMeCD demonstrates that explicit control theory offers a highly efficient, interpretable, and superior path to handling imperfect real-world data.

Summary updated

Abstract

Real-world data is rarely clean; it is plagued by severe class imbalance (long-tailed distributions) and label corruption. Current solutions lean heavily on "black-box" meta-learning to re-weight samples. However, this paradigm introduces a fatal circular dependency: it relies on pristine, balanced validation sets to guide the optimization, which are essentially non-existent in the wild. We propose ProMeCD, a self-referential framework that breaks this dependency by recasting optimization as an autonomous control problem. Instead of training an opaque neural meta-learner, we employ a transparent proportional-integral controller. The system monitors ``cognitive entropy'' that is a metric derived from von Mises-Fisher gradient statistics to assess learning uncertainty. To resolve the scalar ambiguity between tail and noisy samples, ProMeCD employs a decoupled control strategy: it boosts tail classes via integral accumulation of magnitude deficits when directional consistency is high, while suppressing noise via proportional feedback when consistency collapses. Theoretically, we prove that this mechanism guarantees convergence and formally prevents the minority initial drop, ensuring monotonic improvement for rare classes. Crucially, ProMeCD is fully white-box and validation-free. Experiments on CIFAR-LT, iNaturalist, CIFAR-N, mini WebVision, and Clothing1M confirm that ProMeCD is not merely efficient; it outperforms the recent meta-learner FMW-Net by over 10\% in severe imbalance settings, proving that explicit control theory offers a superior path to handling imperfect data.
BibTeX
@inproceedings{zhu2026promecd,
  author = {Yu Zhu},
  title = {{ProMeCD: Unifying Long-Tailed and Noisy Label Learning via White-Box Control}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=GjfStlzQiC},
}
Figure