Learning with Group Noise

Qizhou Wang, Jiangchao Yao, Chen Gong, Tongliang Liu, Mingming Gong, Hongxia Yang, Bo Han

Abstract

Machine learning in the context of noise is a challenging but practical setting to plenty of real-world applications. Most of the previous approaches in this area focus on the pairwise relation (casual or correlational relationship) with noise, such as learning with noisy labels. However, the group noise, which is parasitic on the coarse-grained accurate relation with the fine-grained uncertainty, is also universal and has not been well investigated. The challenge under this setting is how to discover true pairwise connections concealed by the group relation with its fine-grained noise. To overcome this issue, we propose a novel Max-Matching method for learning with group noise. Specifically, it utilizes a matching mechanism to evaluate the relation confidence of each object w.r.t. the target, meanwhile considering the Non-IID characteristics among objects in the group. Only the most confident object is considered to learn the model, so that the fine-grained noise is mostly dropped. The performance on a range of real-world datasets in the area of several learning paradigms demonstrates the effectiveness of Max-Matching.

Introduction: The Challenge of Group Noise

While significant research has addressed pairwise noise—where individual instances are incorrectly labeled—real-world data frequently manifests as "group noise." In this scenario, a group of objects is collectively associated with a target, but only a subset of these objects is truly relevant. For instance, in an image labeled "Toaster," the scene might also contain a sink, a fridge, and a microwave. While the group-level label is accurate, the fine-grained relationship between individual image regions and the label is noisy.

Conceptual illustration of group noise across different learning paradigms: (a) standard pairwise noise, (b) group noise in Multiple-Instance Learning, (c) group noise in Partial Label Learning, and (d) group noise in Recommender Systems.

The authors identify two primary challenges with this data structure:

  1. Fine-grained uncertainty: The group's overall relationship to the target is correct, but individual members may be irrelevant or mismatched.
  2. Non-IID characteristics: Objects within a group are often highly correlated (e.g., candidate names for a face in a TV script), a property often ignored by standard denoising methods that assume independent noise.

The Max-Matching Framework

To address these challenges, the authors propose Max-Matching, an end-to-end framework that employs a "matching and selection" strategy. The architecture consists of two synergistic components:

  • Pair Matching: This module evaluates the direct correspondence between an individual object and the target, calculating a probability score based on their embedding inner products.
  • Group Weighting: This component accounts for the Non-IID nature of the group. It generates a "context vector" for each object—a weighted summary of its peers. Objects that align well with the group context receive higher weights, suggesting they are meaningful components rather than outliers.
The Max-Matching framework architecture, illustrating the interaction between Group Weighting (left) and Pair Matching (right) modules, followed by a selection process.

The final Selection Procedure utilizes a max-pooling operation to identify the single most confident object based on combined scores. By training only on this "max" match, the model effectively filters out group noise and avoids the "averaging" trap that degrades generalization.

Experimental Evaluation

Max-Matching was evaluated across three distinct learning paradigms using diverse real-world datasets:

  • Multiple-Instance Learning (MIL): On the SIVAL dataset for object localization, Max-Matching achieved 36.8% accuracy, outperforming the best instance-level baseline (GICF) by 5.3%.
  • Partial Label Learning (PLL): Across datasets like FG-NET (facial age) and Lost (face naming), the method consistently led. On FG-NET, it surpassed the second-best method by 4.37%.
  • Recommender Systems (RS): Using Amazon click history data, the "MM+" variant showed superior HIT@10 and NDCG@10 scores. Furthermore, a one-week online A/B test on a major e-commerce platform revealed a 10% improvement in Click-Through Rate (CTR) over standard Neural Collaborative Filtering.

Conclusion

The core innovation of Max-Matching lies in leveraging internal group correlations as a filter for noise. By selecting only the most confident match, the model avoids memorizing irrelevant group members. Theoretically, the authors demonstrate that this objective serves as a lower-bound approximation of the true data log-likelihood. Future work will explore optimal weighting between pair and group modules and aim to generalize the framework to handle standard independent label noise.

Summary updated

Abstract

Machine learning in the context of noise is a challenging but practical setting to plenty of real-world applications. Most of the previous approaches in this area focus on the pairwise relation (casual or correlational relationship) with noise, such as learning with noisy labels. However, the group noise, which is parasitic on the coarse-grained accurate relation with the fine-grained uncertainty, is also universal and has not been well investigated. The challenge under this setting is how to discover true pairwise connections concealed by the group relation with its fine-grained noise. To overcome this issue, we propose a novel Max-Matching method for learning with group noise. Specifically, it utilizes a matching mechanism to evaluate the relation confidence of each object w.r.t. the target, meanwhile considering the Non-IID characteristics among objects in the group. Only the most confident object is considered to learn the model, so that the fine-grained noise is mostly dropped. The performance on arange of real-world datasets in the area of several learning paradigms demonstrates the effectiveness of Max-Matching
BibTeX
@inproceedings{wang2021learning,
  author = {Qizhou Wang and Jiangchao Yao and Chen Gong and Tongliang Liu and Mingming Gong and Hongxia Yang and Bo Han},
  title = {{Learning with Group Noise}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i11.17222},
}
Figure