Probability Guided Loss for Long-Tailed Multi-Label Image Classification

Dekun LinAAAI 2023 · DOI

TL;DR

This paper tackles the challenging problem of long-tailed multi-label image classification (LTMLC) by proposing a novel "probability guided loss." Instead of directly adjusting class weights, they focus on controlling the dynamics of predicted probabilities during training, aiming to prevent overfitting and improve performance, especially for rare classes. Their method achieves state-of-the-art results on benchmark LTMLC datasets.

What Problem Does This Paper Address?

Real-world image datasets often exhibit a "long-tailed" distribution, meaning a few classes have many examples (head classes), while most classes have very few (tail classes). This imbalance poses a significant challenge for machine learning models, leading to poor performance on the underrepresented tail classes. Multi-label classification (MLC), where an image can belong to multiple categories simultaneously, adds another layer of complexity. Combining these two challenges, long-tailed multi-label image classification (LTMLC) is a particularly difficult and underexplored area. Existing methods often struggle to effectively balance the learning process across head and tail classes, leading to models that are biased towards the majority classes.

Methods

The core contribution of this paper is a novel "probability guided loss" that manipulates the training process by focusing on the evolution of predicted probabilities. They observe that existing cost-sensitive methods, while effective, influence the predicted probabilities of both positive and negative labels in different ways, impacting overall performance.

Their proposed loss has two main components:

  1. Probability Re-Balancing (PR): This component aims to control the rate at which probabilities change during training. It introduces gradient scaling factors that can be adjusted to slow down the growth of probabilities for classes that are increasing too quickly, or to accelerate those that are lagging. This is a generalization of existing methods like Negative-Tolerance Regularization (NTR).
  2. Adaptive Probability-Aware Focal (APAF): This component is designed to dynamically reduce the gap between the probabilities of positive and negative labels. It uses class-wise predicted probabilities to adaptively adjust gradient scaling factors, aiming to more effectively narrow this gap, especially for tail classes.

These two components are integrated into a unified Probability Guided (PG) loss.

Datasets

The authors evaluate their method on two standard long-tailed multi-label image classification datasets:

  • VOC-LT: Constructed from the Pascal VOC dataset, with classes split into head, medium, and tail based on sample counts.
  • COCO-LT: Constructed from the MS-COCO dataset, also with classes categorized into head, medium, and tail.

These datasets are designed to mimic real-world long-tailed distributions in multi-label settings.

Key Insights and Ideas

  • Focusing on Probability Dynamics: The paper highlights that how probabilities evolve during training is as crucial as the final probability values themselves. Controlling this dynamic can be a more effective way to address class imbalance than simply re-weighting losses.
  • Tail Class Performance is Key: The most significant performance gains are observed on tail classes, underscoring the effectiveness of their approach in addressing the core challenge of long-tailed learning.
  • Generalization of Existing Methods: The Probability Re-Balancing component is shown to be a generalization of prior work like NTR, suggesting a unified framework for probability manipulation.
  • Adaptive Control is Crucial: The adaptive nature of APAF, which adjusts based on current predictions, is key to its success in narrowing the positive-negative probability gap, especially on datasets like COCO-LT where this gap is more pronounced.

Key Results

The proposed Probability Guided (PG) loss achieves state-of-the-art performance on both VOC-LT and COCO-LT datasets.

  • On VOC-LT, the PG loss achieves a total mAP of 80.37%, outperforming the previous best method (DB Focal loss) by 2.1%. The gain is particularly notable on tail classes, with an astonishing 4% mAP improvement.
  • On COCO-LT, the PG loss achieves a total mAP of 54.43%, outperforming DB Focal loss by 1%. Performance gains are observed across all class splits, with notable improvements in medium and tail classes.

Ablation studies demonstrate that both PR and APAF contribute significantly to the overall performance gains.

Limitations and Future Work

  • Computational Cost: While not explicitly detailed as a limitation, the adaptive nature of APAF and the multiple scaling factors in PR might introduce some computational overhead compared to simpler loss functions.
  • Generalizability to Other Tasks: The paper focuses on LTMLC. Future work could explore the applicability of probability-guided losses to other long-tailed learning scenarios or different computer vision tasks.
  • Further Exploration of Adaptive Factors: The authors suggest that PR can also be made adaptive, similar to APAF, which could lead to further improvements. They also note that setting only one scaling factor dynamic might be sufficient, hinting at potential avenues for simplifying the adaptive mechanism.

Summary updated

BibTeX
@inproceedings{lin2023probability,
  author = {Dekun Lin},
  title = {{Probability Guided Loss for Long-Tailed Multi-Label Image Classification}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i2.25244},
}
Figure