Learning with Group Noise

Qizhou Wang, Jiangchao Yao, Chen Gong, Tongliang Liu, Mingming Gong, Hongxia Yang, Bo Han

Abstract

Machine learning performance is heavily dependent on data quality. While "pairwise noise"—incorrect labels for individual instances—is well-studied, "Group Noise" presents a more complex challenge. Group noise occurs when a collection of items is linked to a target, but only a subset of those items is relevant. This paper introduces Max-Matching, a framework presented at AAAI-21 that addresses group noise by utilizing internal group correlations and a max-pooling selection mechanism. Experimental results across Multiple-Instance Learning (MIL), Partial Label Learning (PLL), and Recommender Systems (RS) demonstrate that Max-Matching significantly improves robustness and accuracy compared to existing methods.

Introduction

The standard paradigm for denoising focuses on pairwise relationships, where an individual instance is incorrectly mapped to a label. However, many real-world scenarios involve "parasitic" noise within coarse-grained relationships. We define this as Group Noise.

Comparison of different noise and learning paradigms: (a) Pairwise noise, (b) MIL, (c) PLL, and (d) Recommender Systems.

As illustrated in Figure 1, group noise manifests in several ways:

  • Multiple-Instance Learning (MIL): A "bag" of instances (e.g., image regions) is labeled, but only a few instances contain the target object.
  • Partial Label Learning (PLL): An instance is associated with a set of candidate labels, only one of which is correct.
  • Recommender Systems (RS): A user's interaction history (a group of items) contains both intentional interests and accidental "noisy" clicks.

The primary challenge in these settings is that group members are often non-independently and identically distributed (Non-IID). Traditional methods that treat group members as independent or average their contributions often allow noise to degrade model performance.

The Max-Matching Framework

To tackle group noise, we propose Max-Matching, an end-to-end training approach designed to isolate the true pairwise connection within a group. The architecture consists of two synergistic components: Pair Matching and Group Weighting.

The Max-Matching architecture, illustrating the integration of Group Weighting and Pair Matching modules followed by a selection process.

1. Pair Matching

The Pair Matching module evaluates the direct compatibility between an individual object and the target. By projecting both into a shared embedding space, the model estimates the probability of a match using a Softmax function.

2. Group Weighting

The Group Weighting module serves as the "contextual filter." It analyzes the relationships between objects within the group. The intuition is that relevant objects often share common features or appear in similar contexts within a group labeled with a specific target. By calculating a "context vector," the model assigns higher weights to objects that align with the group's collective signal, effectively down-weighting outliers.

3. Selection via Max-Pooling

Rather than aggregating the entire group's output—which risks incorporating noise—Max-Matching employs a max-pooling operation. The model identifies the single most confident object-target pair based on the combined scores from both modules. Only this "best" pair is used to calculate the loss and update the model parameters, effectively pruning irrelevant data during training.

Experimental Evaluation

The effectiveness of Max-Matching was validated across three distinct domains:

  • Partial Label Learning (PLL): On the FG-NET facial age dataset, Max-Matching achieved 11.0% accuracy, nearly doubling the performance of the next best baseline (CLPL at 6.5%).
  • Multiple-Instance Learning (MIL): On the SIVAL object localization dataset, the method reached 36.8% accuracy, outperforming specialized MIL architectures like IMIL and GICF.
  • Recommender Systems (RS): Tests on Amazon datasets showed significant gains in ranking metrics (HIT@10 and NDCG). Furthermore, a real-world deployment on a major e-commerce platform resulted in a 10% improvement in Click-Through Rate (CTR).

Conclusion

Max-Matching provides a theoretically grounded and practically effective solution for learning under group noise. By leveraging the Non-IID nature of group data and employing a selective training strategy, the framework prevents the model from memorizing irrelevant features. Future work will explore adaptive trade-offs between pair and group weighting and extending the framework to handle hybrid noise scenarios.

Summary updated

Abstract

Machine learning in the context of noise is a challenging but practical setting to plenty of real-world applications. Most of the previous approaches in this area focus on the pairwise relation (casual or correlational relationship) with noise, such as learning with noisy labels. However, the group noise, which is parasitic on the coarse-grained accurate relation with the fine-grained uncertainty, is also universal and has not been well investigated. The challenge under this setting is how to discover true pairwise connections concealed by the group relation with its fine-grained noise. To overcome this issue, we propose a novel Max-Matching method for learning with group noise. Specifically, it utilizes a matching mechanism to evaluate the relation confidence of each object w.r.t. the target, meanwhile considers the Non-IID characteristics among objects in the group. Only the most confident one of the objects is used to learn the model, so that the fine-grained noise is mostly dropped. The performance on a range of real-world datasets in the area of several learning paradigms demonstrates the effectiveness of Max-Matching.
BibTeX
@inproceedings{wang2021learning,
  author = {Qizhou Wang and Jiangchao Yao and Chen Gong and Tongliang Liu and Mingming Gong and Hongxia Yang and Bo Han},
  title = {{Learning with Group Noise}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i11.17222},
}
Figure