Autoregressive Generative Modeling with Noise Conditional Maximum Likelihood Estimation

Henry Li, Yuval Kluger

Abstract

We introduce a simple modification to the standard maximum likelihood estimation (MLE) framework. Rather than maximizing a single unconditional likelihood of the data under the model, we maximize a family of noise conditional likelihoods consisting of the data perturbed by a continuum of noise levels. We find that models trained this way are more robust to noise, obtain higher test likelihoods, and generate higher quality images. They can also be sampled from via a novel score-based sampling scheme which combats the classical covariate shift problem that occurs during sample generation in autoregressive models. Applying this augmentation to autoregressive image models, we obtain 3.32 bits per dimension on the ImageNet 64x64 dataset, and substantially improve the quality of generated samples in terms of the Frechet Inception distance (FID) -- from 37.50 to 12.09 on the CIFAR-10 dataset.

Introduction

Generative modeling has traditionally been divided between Autoregressive (AR) models, which offer exact likelihood estimation, and Diffusion models, which produce high-fidelity samples through iterative refinement. While AR models like PixelCNN are mathematically rigorous, they are notoriously brittle during sampling. This fragility stems from a fundamental pitfall in standard Maximum Likelihood Estimation (MLE): models often over-index on "bit-level" statistics that are visually irrelevant to humans.

A "Sanity Test" reveals this sensitivity: flipping the least significant bit (LSB) of an image's pixels makes it visually identical to a human, yet its likelihood under a standard AR model plummets. This sensitivity leads to a "covariate shift" during autoregressive generation. Because pixels are generated sequentially, a tiny error in an early pixel acts as noise for subsequent ones. Since the model was never trained on noisy data, these errors compound, causing the generation process to drift into nonsensical static.

Methodology: Noise Conditional Maximum Likelihood (NCML)

To bridge the gap between AR and diffusion models, we propose Noise Conditional Maximum Likelihood (NCML). This framework modifies standard MLE training by forcing the model to learn a family of distributions across varying noise levels.

1. Noise-Conditional Training

Instead of maximizing the likelihood of clean data alone, the model is trained to maximize a family of likelihoods $p(x|t)$ conditioned on a noise scale $t$. Here, $t=0$ represents clean data, while higher $t$ values represent data perturbed by varying levels of noise, modeled using Stochastic Differential Equations (SDEs). This training acts as a powerful regularizer, preventing the model from over-indexing on visually irrelevant details and forcing it to focus on global structure.

2. Two-Part Sampling Scheme

NCML enables a novel sampling approach that combines the strengths of AR generation and score-based refinement:

  • Noisy Sampling: The model generates a "draft" image using standard autoregressive sampling at a high noise level ($t > 0$). At these levels, the model is more robust to small errors, mitigating the covariate shift problem.
  • Score-based Refinement: Any differentiable likelihood model is inherently a score-based model. By training on noise, we can calculate the "score function" (the gradient of the log-likelihood) via backpropagation. This score is then used to refine the noisy draft into a clean image through a short reverse-diffusion process.
The NCML framework involves training on a continuum of noise levels and using the learned gradients for score-based refinement during sampling.

Evaluation and Results

We evaluated NCML on standard benchmarks including CIFAR-10, ImageNet 64x64, and CelebA 64x64. The results demonstrate a significant leap in both robustness and image quality.

Quantitative Performance

On the CIFAR-10 dataset, NCML improved the Frechet Inception Distance (FID)—a key metric for image realism—from 37.50 down to 12.09. This performance brings AR models much closer to state-of-the-art diffusion models. Furthermore, the models remained competitive in compression efficiency, achieving 3.32 bits per dimension (bpd) on ImageNet 64x64.

Qualitative Robustness

In the "Sanity Test," NCML models maintained high likelihoods even when images were perturbed by noise, whereas vanilla MLE models failed completely. Visually, the samples generated via NCML are significantly clearer and more coherent than those from standard AR models.

Comparison of generated samples: Standard autoregressive sampling (left) often suffers from covariate shift and structural collapse, whereas NCML (right) produces high-fidelity, coherent images.

Discussion and Future Work

NCML demonstrates that autoregressive models are not inherently limited to low-quality samples; rather, their brittleness is a byproduct of standard MLE training. By exposing the model to "imperfect" noisy data, it learns to recover from the small errors inevitable during sampling.

While promising, challenges remain. Training NCML models is computationally intensive, typically requiring hundreds of thousands of iterations. Future research will focus on optimizing noise schedules and scaling this hybrid approach to high-resolution images, where the compounding error problem in AR models is most severe.

Summary updated

Abstract

We introduce a simple modification to the standard maximum likelihood estimation (MLE) framework. Rather than maximizing a single unconditional likelihood of the data under the model, we maximize a family of \textit{noise conditional} likelihoods consisting of the data perturbed by a continuum of noise levels. We find that models trained this way are more robust to noise, obtain higher test likelihoods, and generate higher quality images. They can also be sampled from via a novel score-based sampling scheme which combats the classical \textit{covariate shift} problem that occurs during sample generation in autoregressive models. Applying this augmentation to autoregressive image models, we obtain 3.32 bits per dimension on the ImageNet 64x64 dataset, and substantially improve the quality of generated samples in terms of the Frechet Inception distance (FID) -- from 37.50 to 12.09 on the CIFAR-10 dataset.
BibTeX
@inproceedings{li2023autoregressive,
  author = {Henry Li and Yuval Kluger},
  title = {{Autoregressive Generative Modeling with Noise Conditional Maximum Likelihood Estimation}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
  url = {https://openreview.net/forum?id=r0JMLPgGXS_},
}
Figure