Detecting out-of-distribution (OOD) data is a crucial capability for robust machine learning systems. A reliable model should know when it is looking at something it hasn't seen before. Normalizing flows (NFs) initially seemed like the perfect tool for this task because, unlike most deep learning models, they calculate the exact probability density $p(x)$ of an input. Theoretically, a model trained on complex natural images like ImageNet should assign a high probability to ImageNet-like photos and a low probability to something completely different, like street view house numbers (SVHN).
However, researchers have discovered a puzzling paradox: normalizing flows often do the exact opposite. A flow trained on complex natural images will frequently assign a higher likelihood to simpler, unrelated datasets. This paper investigates the root causes of this failure, exploring whether it stems from the training objective itself or inherent architectural flaws that make these models "blind" to semantic differences.
The Paradox of Normalizing Flows
The core issue is that normalizing flows—generative models that provide exact likelihoods—paradoxically assign higher probabilities to OOD data than to their own training sets.
As the histogram above illustrates, a model trained on ImageNet assigns significantly higher likelihoods to the SVHN dataset (yellow) than to the actual ImageNet training data (red). This indicates that the model is not learning the high-level semantic concept of "what makes an image belong to ImageNet," but is instead latching onto lower-level features that happen to be highly prevalent in simpler datasets.
Diagnosing the Failure: Lazy Pattern Matchers
To understand why this happens, the authors focused their investigation on affine coupling layers, the foundational building blocks of popular flow models like RealNVP and Glow. These layers operate by splitting the input into two parts; one part remains unchanged and is used to predict the scale ($s$) and shift ($t$) parameters for transforming the second part.
The researchers employed several novel techniques to diagnose the failure, including latent space visualization, architectural ablations, and masking strategy comparisons. Their investigation yielded several key insights:
- Flows are "Lazy" Pattern Matchers: NFs primarily learn local pixel correlations (e.g., "this pixel is usually the same color as its immediate neighbor"). Because simple OOD datasets like SVHN or MNIST have very strong, uniform local correlations (like large patches of solid background color), the flow finds them "easy" to explain and assigns them high likelihoods, regardless of their semantic meaning.
- Coupling Layer Co-adaptation: The authors identified a phenomenon where layers "co-adapt." One layer encodes information about a masked region into the unmasked region, and the next layer decodes it. This allows the model to achieve high likelihoods by essentially "passing notes" between layers, bypassing the need to learn the actual structure of the data.
- Inductive Bias Trumps Objective: The Maximum Likelihood objective doesn't inherently force a model to learn semantics. The "shortcuts" provided by the architecture's inductive biases are what the model actually learns.
Crucially, visualizing the latent space revealed that it is not semantic. As seen in the visualizations above, when an OOD image (like the handwritten '7') is passed through the flow, you can still clearly see its edges and shapes in the latent representation. The flow isn't transforming the image into a meaningful abstract concept; it is merely performing a complex rearrangement of pixels.
Proposed Solutions: Bottlenecks and Embeddings
Armed with an understanding of why flows fail, the authors tested modifications to force the models to learn semantic structure:
- Architectural Bottlenecks: By introducing a low-dimensional bottleneck (dimension $l=10$) into the internal "st-networks" of the coupling layers, the authors restricted the network's capacity to simply "copy" local pixel patterns. This forced the model to stop relying on local shortcuts, significantly improving OOD detection. For a model trained on FashionMNIST, the likelihood gap between in-distribution and OOD (MNIST) data widened substantially.
- Semantic Embeddings: Instead of training on raw pixels, the authors trained flows on high-level semantic embeddings extracted from a pre-trained EfficientNet. This proved to be a highly effective solution. When flows were trained on these embeddings, OOD detection became nearly perfect. For models trained on SVHN and CelebA embeddings, the AUROC (a measure of detection accuracy) reached close to 100%.
Conclusion and Future Directions
This research demonstrates that the properties enabling normalizing flows to generate high-fidelity images—specifically their ability to model local pixel correlations—can have a detrimental effect on OOD detection. Flows are biased toward "simpler" data, and by changing the architecture or the input space to prevent the model from exploiting this simplicity, they can be transformed into highly effective OOD detectors.
However, limitations remain. The most successful method requires a pre-trained supervised model (EfficientNet) to generate embeddings, which may not be available in all domains. Furthermore, while this paper focuses heavily on affine coupling layers, other types of flows might exhibit different failure modes. Future work will need to focus on designing flow architectures that inherently possess "semantic" inductive biases without relying on external pre-trained encoders.
Summary updated