CNT (Conditioning on Noisy Targets): A new Algorithm for Leveraging Top-Down Feedback

Alexia Jolicoeur-Martineau, Alex Lamb, Vikas Verma, Aniket DidolkararXiv 2022 · DOI · arXiv

Abstract

Conditioning on Noisy Targets (CNT) is a novel regularization technique designed to improve supervised learning by integrating top-down feedback during training. By conditioning a model on a target label corrupted by varying levels of noise, CNT creates a continuous curriculum that spans from trivial "hint-based" tasks to the standard objective. This approach significantly enhances generalization, particularly in low-capacity models and reinforcement learning environments. We demonstrate the efficacy of CNT across diverse benchmarks, including ImageNet classification, relational reasoning in Sort-of-CLEVR, and Atari game performance using Decision Transformers.

Introduction

Standard deep learning models typically operate via bottom-up processing, attempting to map complex raw inputs to high-level targets in a single leap. This differs significantly from human learning, which is often progressive and guided by top-down feedback—where high-level knowledge helps resolve low-level ambiguity. While "Curriculum Learning" attempts to replicate this by ordering training data from easy to hard, it is notoriously difficult to implement because it requires a predefined metric for task difficulty.

CNT addresses these challenges by providing a continuous "hint" mechanism that simplifies the learning objective. By feeding a noisy version of the target back into the network, the model learns to solve a spectrum of tasks simultaneously, ranging from easy (low noise) to hard (pure noise).

Methodology

The core of the CNT algorithm is the conditioning of the model on the target label $y$, corrupted by varying levels of noise $t$.

1. The Noise Process

The authors utilize a Variance Preserving (VP) diffusion process to corrupt the target $y$. At noise level $t=0$, the target is perfectly preserved. As $t$ increases toward $1$, the target is progressively replaced by Gaussian noise until it becomes pure noise at $t=1$.

2. Training Objective and Multi-Scale Learning

During training, the model is presented with the input $x$, the noise level $t$, and the noisy target $y(t)$. The objective is to predict the original, clean target $y(0)$. Rather than following a complex schedule that increases difficulty over time, the authors employ Multi-Scale Learning, where $t$ is sampled uniformly from $[0, 1]$ for every mini-batch. This forces the model to learn the relationship between inputs and targets across all difficulty levels at once.

3. Conditioning Architecture

To inject the "hint" into the network, the authors use conditional normalization layers (similar to FiLM). The noise level $t$ and noisy label $y(t)$ are processed through small embedding networks. These networks produce scale and shift parameters that modulate the feature maps within the main network's normalization layers, effectively providing top-down guidance to the feature extraction process.

4. Inference

At test time, the true target is unknown. To perform a standard prediction, the model is simply provided with pure noise ($t=1$). Because the model was trained to solve the task even when the hint was uninformative (pure noise), it can generalize to perform the task using only the input $x$.

Experimental Evaluation

The authors evaluated CNT across a diverse range of benchmarks, demonstrating its versatility across different architectures and domains.

  • Image Classification: On CIFAR-100, a ResNet18 using CNT achieved 80.33% accuracy, outperforming the 79.93% baseline.
  • Low-Capacity Models: The most significant gains were observed in constrained environments. In a ResNet9 with only 8 channels on TinyImageNet, the baseline accuracy was 19.05%. CNT boosted this to 33.45%, a massive relative improvement that suggests CNT helps small models focus limited parameters on the most relevant features.
  • Relational Reasoning: Using the Sort-of-CLEVR dataset, which requires answering questions about object relationships, CNT improved ResNet performance on the hardest "Binary" questions from 78.34% to 90.73%.
  • Reinforcement Learning: When integrated into the Decision Transformer architecture for Atari games, CNT showed notable improvements. On Seaquest, for instance, the score increased from 894.67 to 1106.75.

Discussion and Insights

Learning to "Cheat"

At low noise levels, the model can essentially "cheat" by utilizing the provided hint. This allows the network to identify useful features in the input $x$ much faster than if it were searching blindly. This acts as a natural curriculum that does not require manual data sorting.

Top-Down Guidance

By conditioning on the label, the model learns how high-level categories should manifest in low-level features. This effectively trains a "top-down" pathway that persists and aids the model's internal logic even when the explicit hint is removed at inference time.

Limitations and Future Work

While CNT shows strong empirical results, several areas remain for future exploration:

  1. Semantic Noise: Currently, noise is added to the target space (e.g., one-hot vectors). Injecting noise into a semantically meaningful embedding space might further improve results by ensuring that noisy "cat" labels remain closer to "dog" than to "car."
  2. Conditioning Methods: This study focused on conditional normalization. Other methods, such as cross-attention in Transformers, may offer different advantages for merging the hint with the input.
  3. Theoretical Foundation: There is currently a gap in theoretical evidence explaining why multi-scale training (training on all noise levels simultaneously) generalizes better than traditional progressive learning schedules.

Summary updated

Abstract

We propose a novel regularizer for supervised learning called Conditioning on Noisy Targets (CNT). This approach consists in conditioning the model on a noisy version of the target(s) (e.g., actions in imitation learning or labels in classification) at a random noise level (from small to large noise). At inference time, since we do not know the target, we run the network with only noise in place of the noisy target. CNT provides hints through the noisy label (with less noise, we can more easily infer the true target). This give two main benefits: 1) the top-down feedback allows the model to focus on simpler and more digestible sub-problems and 2) rather than learning to solve the task from scratch, the model will first learn to master easy examples (with less noise), while slowly progressing toward harder examples (with more noise).
BibTeX
@misc{jolicoeurmartineau2022cnt,
  author = {Alexia Jolicoeur-Martineau and Alex Lamb and Vikas Verma and Aniket Didolkar},
  title = {{CNT (Conditioning on Noisy Targets): A new Algorithm for Leveraging Top-Down Feedback}},
  year = {2022},
  eprint = {2210.09505},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2210.09505},
}
Figure