Introduction
Invertible neural networks (INNs) have become essential tools in modern machine learning, particularly for generative modeling via normalizing flows and memory-efficient training through architectures like RevNets. A critical subclass of these models is measure-preserving networks, which maintain a constant Jacobian determinant of $\pm 1$. This property ensures that the "volume" of the data space remains invariant under transformation.
Despite their practical success, a fundamental theoretical question remained: do the strict structural constraints required for volume preservation limit the expressivity of these networks? While standard Universal Approximation Theorems exist for unconstrained networks, they do not directly apply to architectures like NICE or RevNets. This research provides the first rigorous mathematical proof that measure-preserving neural networks are indeed universal approximators for any dimension $D \geq 2$.
Theoretical Framework: A Dynamical Systems Approach
The authors analyze architectures built from additive coupling layers, the core components of NICE and RevNets. These layers transform inputs by splitting them into two parts and applying a shift to one part based on a function of the other. To prove universal expressivity, the paper adopts a dynamical systems perspective, treating the network transformation as a "flow" of a differential equation.
The proof follows a logical progression: measure-preserving maps are shown to correspond to the flows of divergence-free vector fields, which are then decomposed into simpler Hamiltonian components. The authors demonstrate that additive coupling layers can approximate these components, and by composing these layers, the network can reconstruct the flow of any complex divergence-free system. This framework establishes that if a hypothesis space contains basic coupling modules, it inherits universal approximation capabilities for measure-preserving maps.
Numerical Validation: Learning Physical Flows
While the paper is primarily theoretical, the authors validate their findings through numerical experiments. They simulated a 4D divergence-free dynamical system representing a single charged particle in an electromagnetic field, governed by the Lorentz force. The goal was to train a measure-preserving network to learn the underlying physics (the flow map) from a single trajectory.
With only 200 training samples, the network successfully reconstructed the training data and demonstrated high accuracy in predicting future states. The results shown in Figure 1 confirm that the theoretical capability of these networks translates into practical learnability, even for complex physical systems.
Key Results and Implications
The main contribution is Theorem 1, which states that for a compact set $U \subset \mathbb{R}^D$ ($D \geq 2$), measure-preserving neural networks can approximate any bounded, injective, measure-preserving map $\psi$ in the $L^p$-norm. A significant corollary is that any continuously differentiable injective map with a Jacobian determinant of $\pm 1$ can be approximated.
The necessity of $D \geq 2$ is a crucial insight; in one dimension, measure-preserving maps are restricted to simple translations or reflections, lacking the richness required for universal approximation. This work bridges the gap between deep learning architecture and classical mechanics, providing a physical intuition for the efficacy of additive coupling layers.
Conclusion and Future Work
Several open questions remain. The current proof focuses on the $L^p$-norm (average error), and extending this to the $C$-norm (uniform error) is a subject for future research. Additionally, while the paper proves that approximation is possible, it does not yet quantify the specific network depth or width required to achieve a target precision. Future directions include exploring the approximation capabilities of other structured networks, such as those preserving symplectic or orthogonal properties.
Summary updated