Generative Modelling With Inverse Heat Dissipation

Severi Rissanen, Markus Heinonen, Arno Solin

Abstract

This paper introduces the Inverse Heat Dissipation Model (IHDM), a novel generative framework that constructs images by stochastically reversing the heat equation. Unlike standard diffusion models that rely on Gaussian noise to erase information, IHDM utilizes a "melting" process that progressively blurs images, thereby explicitly leveraging the multi-scale structure inherent in natural data. Our results demonstrate that IHDM achieves competitive image quality while offering unique advantages, such as the disentanglement of global color from fine-grained shape and superior performance in few-shot learning scenarios.

Introduction

Recent advancements in generative modeling have been dominated by diffusion models, which learn to reverse a process of gradual noise injection. However, these models often treat pixels as independent entities, failing to incorporate the multi-scale nature of images where global structures are typically more robust than fine details. The authors propose a shift from noise-based erasure to resolution-based erasure, asking whether a diffusion-like process can be defined by decreasing resolution rather than adding noise.

Methods

The core of the IHDM is the replacement of stochastic noise with deterministic Heat Dissipation.

Forward Process: Information Melting

The forward process is governed by the heat equation—a partial differential equation (PDE) defined as $\frac{\partial \mathbf{u}}{\partial t} = \Delta \mathbf{u}$. This process locally averages pixel values, effectively "melting" the image. Fine details are removed first, eventually leaving only a flat surface representing the mean color of the original image. To ensure computational efficiency, the authors project images into frequency space using the Discrete Cosine Transform (DCT), allowing the heat equation to be solved analytically and rapidly.

The information melting forward process (top) and the generative reverse sequence (bottom) for different image classes.

Reverse Process: Generative Inverse

To generate images, the model learns to stochastically reverse this dissipation. By introducing a small amount of noise during training, the authors ensure the reverse process remains a well-defined Markov chain. The architecture employs a standard U-Net, optimized with a loss function that encourages step-by-step deblurring.

Key Insights and Comparison

A primary contribution of this work is the explicit multi-scale inductive bias. While standard diffusion models implicitly prioritize coarse features because noise affects high-frequency details first, IHDM makes this explicit by directly decaying those frequencies via the heat equation.

Comparison between the standard diffusion model (noise-based) and the inverse heat dissipation model (blur-based).

This approach leads to several key properties:

  • Disentanglement: The process naturally separates global features (like average color) from local textures. By fixing the sampling noise and varying the initial "blurry" state, the model can generate variations of the same object with different color schemes.
  • Data Efficiency: IHDM demonstrates remarkable few-shot generalization. In tests with only 20 MNIST samples, it generated plausible new variations, whereas standard models tended to memorize the training set.

Results and Discussion

The model was evaluated on benchmarks including MNIST, CIFAR-10, and high-resolution datasets like FFHQ. IHDM achieved FID scores of 18.96 on MNIST and 45.06 on CIFAR-10. While these scores currently trail the most highly optimized diffusion models, they validate the "inverse heat" approach as a viable alternative.

Qualitatively, the model generates complex architectural and facial images with a clear hierarchical structure. Large-scale layouts are established in early generative steps, with fine-grained textures added progressively. Future work will focus on optimizing architectures specifically for deblurring and exploring applications in other hierarchical domains like audio or graph data.

Summary updated

BibTeX
@inproceedings{rissanen2023generative,
  author = {Severi Rissanen and Markus Heinonen and Arno Solin},
  title = {{Generative Modelling With Inverse Heat Dissipation}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
  url = {https://openreview.net/forum?id=4PJUBT9f2Ol},
}
Figure