Training image classifiers using Semi-Weak Label Data

Ankit Parag Shah, Bhiksha RajICLR 2023 · OpenReview

TL;DR

This paper introduces "semi-weak labels" for image classification, where we know not just if a class is present, but also its exact count within an image (or "bag"). They propose a novel three-stage framework that leverages this count information to achieve performance comparable to fully supervised methods, significantly outperforming traditional weakly supervised approaches.

What Problem Does This Paper Address?

Traditional supervised learning requires every single instance (e.g., every dog in an image) to be precisely labeled. This is often prohibitively expensive and time-consuming. "Weakly supervised" learning offers a compromise by only indicating the presence or absence of a class within a larger group of instances (a "bag," like an entire image). However, this still leaves a performance gap compared to fully supervised methods. The authors identify a middle ground: "semi-weak labels." This type of labeling is often much easier for humans than precise bounding boxes, as it only requires counting occurrences (e.g., "there are 3 dogs in this image"). The problem is how to effectively train classifiers using this richer, yet still imperfect, count information.

Methods

The paper proposes a three-stage framework:

  1. Estimating Class Counts: This stage uses a Poisson loss function to predict the expected count of each class within a bag. The Poisson distribution is chosen because it naturally models discrete, non-negative counts. This stage also incorporates a classification loss and L1 regularization to handle sparse bags and improve inter-class separability.
  2. Translating Expected Counts to Exact Counts: Since the first stage outputs expected (real-valued) counts, this stage converts them into exact (integer) counts. They employ a greedy algorithm that iteratively assigns counts to classes based on maximizing the likelihood of the Poisson distribution, ensuring the total count matches the bag size.
  3. Instance-Level Label Assignment (Decoder): Given the exact counts and the instance-level predictions from a base network (e.g., a ResNet), this stage uses a linear-sum assignment problem (solved via a greedy approach) to assign each instance within the bag to a specific class, effectively inferring instance-level labels.

Datasets

The experiments were conducted using the CIFAR-10 dataset. The authors generated custom datasets by creating "bags" of images with varying sizes and different distributions of class instances (Poisson, exponential, and uniform). These generated datasets are summarized in Table 1 of the paper.

Key Insights and Ideas

  • The Power of Counting: Simply knowing the count of a class in a bag, rather than just its presence, provides a significant signal that can bridge the gap between weak and full supervision.
  • Poisson Distribution is Robust: The Poisson loss function, designed for count data, proved effective even when the underlying data distribution wasn't strictly Poisson, suggesting its robustness.
  • Greedy Assignment Works (Mostly): While a greedy approach for instance-level assignment is used, the paper highlights that a dedicated "decoder" stage (the third stage) significantly improves performance, especially for sparse bags where greedy methods can become unstable.
  • Semi-Weak Labels are Practical: The authors argue that semi-weak labels are often more natural and less effortful to obtain than precise instance-level labels, making this paradigm highly relevant for real-world applications.

Key Results

The proposed semi-weak label learning framework achieved results comparable to fully supervised methods on the CIFAR-10 dataset. For instance, on dataset p2, their framework achieved 91.21% instance-level precision, very close to the fully supervised upper-bound. Crucially, it significantly outperformed the weakly supervised baseline (which achieved 87% instance-level precision) and the learning-from-proportion baseline. Ablation studies showed that each component of their three-stage framework contributed to the overall performance, with the decoder stage providing a notable boost.

Limitations and Future Work

The paper acknowledges that the greedy assignment in the decoder can become unstable for very sparse bags, though the decoder stage helps mitigate this. They suggest that their work can be extended to other types of data beyond images, and that semi-weak labels offer a promising avenue for real-world tasks where counting information is readily available.

Summary updated

Abstract

This paper introduces a new semi-weak label learning paradigm which provides additional information in comparison to the weak label classification. We define semi-weak label data as data where we know the presence or absence of a given class and additionally we have the information about the exact count of each class as opposed to knowing the label proportions. A three-stage framework is proposed to address the problem of learning from semi-weak labels. It leverages the fact that counting information is naturally non-negative and discrete. Experiments are conducted on generated samples from CIFAR-10 and we compare our model with a fully-supervised setting baseline, a weakly-supervised setting baseline and a learning from proportion(LLP) baseline. Our framework not only outperforms both baseline models for MIL-based weakly supervised setting and learning from proportion setting, but also gives comparable results compared to the fully supervised model. Further, we conduct thorough ablation studies to analyze across datasets and variation with batch size, losses architectural changes, bag size and regularization, thereby demonstrating robustness of our approach.
BibTeX
@inproceedings{shah2023training,
  author = {Ankit Parag Shah and Bhiksha Raj},
  title = {{Training image classifiers using Semi-Weak Label Data}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
  url = {https://openreview.net/forum?id=kPL4YzdDqWE},
}

Loading related content...

Figure