Do We Always Need to Penalize Variance of Losses for Learning with Label Noise?
Lists
Only you see this
Abstract
Algorithms which minimize the averaged loss have been widely designed for dealing with noisy labels. Intuitively, when there is a finite training sample, penalizing the variance of losses will improve the stability and generalization of the algorithms. Interestingly, we found that the variance of losses sometimes needs to be increased for the problem of learning with noisy labels. Specifically, increasing the variance of losses would boost the memorization effect and reduce the harmfulness of incorrect labels. Regularizers can be easily designed to increase the variance of losses and be plugged in many existing algorithms. Empirically, the proposed method by increasing the variance of losses could improve the generalization ability of baselines on both synthetic and real-world datasets.
BibTeX
@inproceedings{lin2023do,
author = {Yexiong Lin and Yu Yao and Yuxuan Du and Jun Yu and Bo Han and Mingming Gong and Tongliang Liu},
title = {{Do We Always Need to Penalize Variance of Losses for Learning with Label Noise?}},
booktitle = {International Conference on Learning Representations},
year = {2023},
url = {https://openreview.net/forum?id=FJdSi_seSg},
}Loading related content...