Tractable Density Estimation on Learned Manifolds with Conformal Embedding Flows

Brendan Leigh Ross, Jesse C. Cresswell

Abstract

Conformal Embedding Flows (CEFs) enable exact, tractable density estimation for data residing on low-dimensional manifolds. By restricting the mapping from latent space to data space to be "conformal" (angle-preserving), the authors simplify the complex Jacobian calculations usually required for injective flows. This allows for efficient joint training of both the manifold structure and the data distribution, scaling to high-dimensional image data.

Introduction

Normalizing Flows (NFs) are a powerful class of generative models because they provide exact likelihoods. However, traditional NFs require the latent space to have the same number of dimensions as the data space. This is a significant limitation because real-world data, such as images, typically lives on a low-dimensional manifold embedded in a high-dimensional space. For instance, a $64 \times 64$ image exists in a 4,096-dimensional pixel space, but the set of meaningful images occupies a much smaller, curved subset.

Previous attempts to build "injective flows"—which map a small latent space to a large data space—often sacrificed the ability to compute exact densities or required computationally expensive approximations. This paper introduces CEFs to recover tractable likelihoods while respecting the low-dimensional nature of the data.

Methodology

The CEF architecture: mapping from latent space $\mathcal{Z}$ to $\mathcal{U}$ via $\mathbf{h}$, then to the data manifold $\mathcal{M}$ in $\mathcal{X}$ via the conformal embedding $\mathbf{g}$.

The core innovation is the use of conformal embeddings. A mapping is conformal if it preserves angles locally, meaning its Jacobian matrix $J_g$ satisfies $J_g^T J_g = \lambda^2(u) I$ for some scalar factor $\lambda$. In the generalized change-of-variables formula for density estimation, one must compute $\det(J_g^T J_g)$. For general injective mappings, this is computationally prohibitive; for conformal mappings, it collapses into a simple power of the scalar $\lambda$.

The authors propose several building blocks for these flows:

  • Conformal Mappings: Compositions of translations, orthogonal transformations, and Special Conformal Transformations (SCTs).
  • Orthogonal $k \times k$ Convolutions: A new layer type that scales orthogonal transformations to image data.
  • Piecewise Conformal Embeddings: Relaxing the condition to be conformal "almost everywhere" to increase expressivity, analogous to the use of ReLU in standard networks.
  • Joint Training: The tractability of the density allows the model to be trained using a mixed loss function that optimizes for both reconstruction (manifold learning) and maximum likelihood (distribution modeling) simultaneously.

Experimental Evaluation

Density estimation and sampling results on a synthetic 2D sphere manifold.

The authors evaluated CEFs on several datasets:

  • Spherical Data: A synthetic distribution supported on a 2D sphere embedded in 3D space, demonstrating the model's ability to learn curved manifolds.
  • Synthetic CIFAR-10 Ship Manifolds: Data generated by a GAN where the true intrinsic dimensionality is known (64 and 512 dimensions).
  • MNIST and CelebA: Classic image datasets representing complex, high-dimensional natural data.

Key Results and Insights

  • The "Conformal" Sweet Spot: Conformal constraints are restrictive enough to make the math tractable but flexible enough to learn complex, curved manifolds. While Liouville’s theorem limits conformal mappings in higher dimensions, the authors show that conformal embeddings (low-to-high dim) are significantly more expressive.
  • Joint vs. Sequential Training: Joint training proved superior to sequential two-step processes, especially for complex data like CelebA, as it allows the manifold to adapt to the needs of the density estimator.
  • Scalability: CEFs are the first injective flow models to provide exact log-determinant computation that scales to high-dimensional image data. On CelebA, the model achieved competitive FID scores and reconstruction losses comparable to state-of-the-art non-conformal flows.

Conclusion and Limitations

While CEFs provide a scalable path to exact density estimation on manifolds, they face an expressivity trade-off. They are currently less flexible than models like Neural Spline Flows, which lack tractable densities. Future work may involve discovering more expressive conformal layers or relaxing the constraint without losing computational benefits. Additionally, the current topology of the learned manifold is restricted to Euclidean space, which may not suit data with disconnected components.

Summary updated

BibTeX
@inproceedings{ross2021tractable,
  author = {Brendan Leigh Ross and Jesse C. Cresswell},
  title = {{Tractable Density Estimation on Learned Manifolds with Conformal Embedding Flows}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=DqU-rIHy4Eh},
}

Loading related content...

Figure