TL;DR
Conformal Embedding Flows (CEFs) allow generative models to perform exact density estimation on data that lives on low-dimensional manifolds (like images) rather than in full-dimensional space. By restricting the manifold-learning layers to be "conformal"—meaning they preserve angles locally—the authors make the complex math of non-square Jacobian determinants computationally trivial, enabling joint training of both the manifold and the data density.
What Problem Does This Paper Address?
Normalizing Flows (NFs) are powerful because they provide an exact "likelihood"—a measure of how well the model fits the data. However, standard NFs assume that the data fills the entire high-dimensional space it sits in. In reality, data like images usually lives on a much lower-dimensional "manifold" (e.g., the set of all possible faces is a tiny fraction of the set of all possible pixel combinations).
When you try to fit a standard NF to manifold-supported data, the density can explode to infinity, or the model might waste parameters trying to describe "meaningless noise" in the empty space. Previous attempts to build "Injective Flows" (flows that map from low to high dimensions) either required expensive approximations to calculate density or gave up on exact density estimation entirely.
Methods
The authors introduce Conformal Embedding Flows (CEFs). The core idea is to decompose the generative process into two parts:
- A standard bijective flow ($h$): This learns the complex density within a low-dimensional latent space.
- A conformal embedding ($g$): This maps that latent space into the high-dimensional data space.
The "secret sauce" is the conformal condition. Mathematically, a mapping is conformal if its Jacobian $J$ satisfies $J^T J = \lambda^2 I$, where $\lambda$ is a scalar factor. This is brilliant because the term needed for density estimation—the determinant of the non-square Jacobian—simplifies from a nightmare of matrix operations into a simple power of the scalar $\lambda$.
To make this work for deep learning, the authors developed several new "conformal building blocks":
- Orthogonal $k \times k$ Convolutions: A way to scale conformal mappings to image data.
- Special Conformal Transformations (SCTs): Non-linear layers borrowed from conformal field theory.
- Conditional Orthogonal Transformations: Piecewise layers that increase the model's expressivity.
Datasets
The researchers evaluated CEFs on a mix of synthetic and real-world benchmarks:
- Spherical Data: A 2D distribution wrapped around a 3D sphere to test manifold recovery.
- Synthetic Image Manifolds: High-dimensional "ship" images generated from a pretrained StyleGAN2-ADA (64 and 512 dimensions).
- Natural Images: The standard MNIST (handwritten digits) and CelebA (celebrity faces) datasets.
Key Insights and Ideas
- Conformality is the "Sweet Spot": While requiring orthogonal columns in a Jacobian is enough for a single layer, it isn't "composable" (the property breaks when you stack layers). Enforcing the stricter conformal condition ensures that the entire stack remains mathematically tractable.
- Joint Training is Better: Because the density is exact and tractable, CEFs can be trained using a "mixed loss" that optimizes for both reconstruction (getting the manifold right) and maximum likelihood (getting the density right) simultaneously.
- The "Aha!" Moment on CelebA: On complex datasets like CelebA, the authors found that joint training actually helps the model find a better manifold for density estimation. Sequential training (fixing the manifold first, then learning the density) often leaves the flow struggling to fit a poorly-initialized space.
Key Results
-
Spherical Recovery: The CEF successfully learned both the 2D surface of a sphere and the complex probability density on that surface. CEF successfully learning a density on a 3D sphere.
-
CelebA Performance: On the high-dimensional CelebA dataset, the jointly-trained CEF produced significantly better samples (lower FID score) than sequentially-trained versions.
- Competitive Generative Quality: While CEFs use a more restricted architecture than some other injective flows, they remain competitive in terms of image fidelity and diversity, while offering the unique advantage of exact, tractable likelihoods. Uncurated samples from CelebA showing the quality of the joint CEF model.
Limitations and Future Work
The main limitation is expressivity. Liouville’s theorem states that conformal mappings in dimensions higher than two are quite restricted (mostly combinations of rotations, scalings, and inversions). While the authors introduced piecewise and conditional layers to bypass some of this, CEFs are still less flexible than "standard" non-conformal neural networks.
Future work will likely focus on discovering more expressive conformal building blocks and addressing topological constraints (e.g., the current model assumes the data manifold is "flat" like Euclidean space, which might not be true for data with holes or complex loops).
Summary updated