Learning Stationary Markov Processes with Contrastive Adjustment

Ludvig Bergenstråhle, Jens Lagergren, Joakim LundebergarXiv 2023 · DOI · arXiv

Introduction

Generative modeling has achieved significant milestones through Generative Adversarial Networks (GANs), Diffusion Models, and Autoregressive models. However, these architectures often face trade-offs between training stability, sampling speed, and the ability to perform local exploration on the data manifold. This paper introduces contrastive adjustment, a novel optimization algorithm designed to train Markov transition kernels so that their stationary distribution matches a target data distribution. By learning a stationary Markov chain, the proposed approach enables high-quality image synthesis, efficient generation of variants, and flexible inpainting across both continuous and categorical domains.

Methodology: Contrastive Adjustment

The core innovation of this work is an algorithm that focuses on the detailed balance criterion rather than direct density estimation. Traditional Markov Chain Monte Carlo (MCMC) models often struggle to ensure the learned transition operator leads to the desired stationary state.

The Adjustment Mechanism

The contrastive adjustment algorithm iteratively refines a transition kernel $p(y|x)$. If a forward transition from $x$ to $y$ is found to be more probable than the reverse transition from $y$ to $x$ (relative to the underlying data distribution), the model decreases the probability of that forward step. In a steady state, these upward and downward adjustments cancel out, ensuring the model has successfully captured the data distribution as its stationary state.

Visualizing the contrastive adjustment process where transition probabilities are shifted to satisfy detailed balance.

Noise Kernels and Multi-Scale Modeling

To navigate complex, high-dimensional data manifolds, the authors utilize "noise kernels" inspired by diffusion models. These kernels operate across multiple noise levels: high-noise kernels facilitate rapid movement across the data space, while low-noise kernels refine fine-grained details. This framework is uniquely versatile, supporting both continuous data (via Gaussian noise) and categorical data (via absorbing state noise), providing a unified path for disparate data types.

Experimental Setup

The authors evaluated the contrastive adjustment framework on three standard image synthesis benchmarks to demonstrate its scaling and versatility:

  • CIFAR-10: $32 \times 32$ images, used for both continuous and discretized categorical experiments.
  • LSUN Church: $128 \times 128$ images focusing on architectural structures.
  • CelebA-HQ: High-resolution $256 \times 256$ face images to test high-fidelity synthesis.

Key Results and Insights

The model demonstrates competitive performance across all benchmarks, particularly excelling in its ability to generate local variants and perform zero-shot tasks.

Synthesis results across CelebA-HQ, LSUN Church, and CIFAR-10 showing high-fidelity generation and diversity.

Quantitative Performance

On CIFAR-10, the model achieved an FID score of 18.27, outperforming several established energy-based models. In the categorical domain, it reached an FID of 14.76, proving robustness for discrete data. High-resolution scaling was also successful, with the LSUN Church model achieving a competitive FID of 9.40.

Stationarity and Local Exploration

A unique advantage of this approach is that the learned Markov chain is stationary. Users can start the chain with a real image and run it for a few steps to generate "variants." This allows for local exploration of the data manifold—such as generating different versions of a specific face or building—which is a powerful tool for design and human-in-the-loop applications.

Zero-Shot Inpainting

Without specific training for image completion, the noise kernels were able to perform high-quality inpainting. By simply conditioning the denoising distribution on known pixels, the model naturally fills in missing regions while maintaining consistency with the learned stationary distribution.

Discussion and Future Work

While contrastive adjustment provides a simple gradient-based update rule that avoids explicit density calculation, some challenges remain. Sampling from scratch requires running the Markov chain for 100–500 steps, which is slower than single-pass models like GANs. Additionally, while empirical evidence suggests the algorithm is well-behaved, formal mathematical proofs regarding convergence rates are still needed. Future research may expand this framework into non-image domains such as text and audio, where the ability to handle discrete categorical data is particularly advantageous.

Summary updated

BibTeX
@misc{bergenstrahle2023learning,
  author = {Ludvig Bergenstråhle and Jens Lagergren and Joakim Lundeberg},
  title = {{Learning Stationary Markov Processes with Contrastive Adjustment}},
  year = {2023},
  eprint = {2303.05497},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2303.05497},
}
Figure