Enhancing Robustness of Deep Networks Based on a Two-phase Model of Their Training with Noisy Labels
Lists
Only you see this
Abstract
In this study we model explicitly the learning behavior of deep neural networks (DNNs) trained with noisy labels in image classification. Specifically, we show theoretically and experimentally that the training process can be divided into two phases: a learning phase in which the outputs of DNNs converge to a hidden noise distribution no matter whether the training samples are clean or noisy; and a memorization phase in which DNNs start to overfit until the output for each training sample converges to its corresponding noisy one-hot labels. This two-phase model enables us to develop two simple yet accurate methods that rely on the outputs of DNNs to estimate the noise transition matrix (NTM). It also enables us to resolve a pitfall of many existing methods for robust training under noisy labels based on the small-loss assumption, namely clean samples have smaller loss than noisy samples in the early training phase. We show that these methods fail when NTM is not a column diagonally-maximum matrix and that this pitfall can be fixed by modifying the small-loss assumption based on our NTM estimation methods.
BibTeX
@misc{luo2022enhancing,
author = {Yaoru Luo and Ge Yang},
title = {{Enhancing Robustness of Deep Networks Based on a Two-phase Model of Their Training with Noisy Labels}},
year = {2022},
url = {https://openreview.net/forum?id=ay4xkpMnyE},
howpublished = {OpenReview},
}Loading related content...