Enhanced Importance Sampling Through Latent Space Exploration in Normalizing Flows

Liam Anthony Kruse, Alexandros E. Tzikas, Harrison Delecki, Mansur M. Arief, Mykel J. KochenderferAAAI 2025 · DOI

Abstract

Estimating the probability of rare failure events is a critical challenge in safety-critical engineering, such as autonomous vehicle and aircraft testing. Traditional Importance Sampling (IS) often fails when failure regions are complex or disconnected. This paper introduces a novel approach that performs IS within the latent space of a pre-trained Normalizing Flow (NF). By mapping complex simulation outcomes into a simplified, isotropic latent representation, the proposed method achieves superior coverage of failure modes and significantly higher sample efficiency across robotics benchmarks.

Introduction

In safety-critical domains, proving a system's reliability requires identifying "rare events"—failures that occur with extremely low probability. Standard Monte Carlo simulations are computationally prohibitive for these tasks. Importance Sampling (IS) offers a statistical shortcut by using a "proposal distribution" to over-sample likely failure regions. However, when failure regions are non-convex, disconnected, or highly warped (e.g., "banana-shaped"), standard IS algorithms often suffer from "mode dropping," leading to dangerously optimistic safety estimates.

Methodology: Latent Space Importance Sampling

The core innovation of this work is leveraging Normalizing Flows (NFs) to "un-warp" the search space for rare events. NFs learn a reversible mapping $T$ between a complex target distribution $x$ (simulation outcomes) and a simple latent distribution $u$ (typically a standard Gaussian).

Conceptual overview of mapping complex target space failure regions into a simplified latent space for more effective proposal fitting.

As illustrated in Figure 1, the process involves four key steps:

  1. Inverse Mapping: Known failure points are mapped back to the latent space using the inverse flow $u = T^{-1}(x)$.
  2. Ellipsoidal Approximation: In the latent space, the researchers compute a Löwner–John ellipsoid—the minimum volume ellipsoid containing the failure points—to guide the search.
  3. Latent Search: Standard IS algorithms, such as Cross-Entropy (CE) and Sequential Importance Sampling (SIS), are applied within this simplified, isotropic latent geometry.
  4. Forward Mapping: Samples generated from the refined latent proposal are pushed through the forward flow $x = T(u)$ to generate realistic simulation scenarios.

Experimental Validation

The methodology was evaluated on three high-dimensional robotics benchmarks:

  • Nonholonomic Robot (3D): A mobile robot with multimodal failure regions.
  • Cornering Racecar (8D): A vehicle model where "spinning out" represents a rare failure.
  • F-16 Ground Collision Avoidance (12D): A high-fidelity jet fighter model executing recovery maneuvers.

Key Results and Discussion

The latent-space methods, termed Latent-CE and Latent-SIS, consistently outperformed their target-space counterparts in accuracy, coverage, and efficiency.

Quantitative comparison of Latent-space vs. Target-space methods across accuracy, coverage, and sample efficiency metrics.

As shown in the results table (Figure 6), target-space methods often failed completely in high-dimensional tasks like the F-16 simulation, yielding a relative error of -1.000 (indicating they missed the failure region entirely). In contrast, Latent-CE provided accurate estimates and achieved significantly higher coverage scores. Furthermore, efficiency gains were substantial; for the F-16 task, Latent-CE converged in 400,000 samples, whereas the target-space version required 900,000 samples while still failing to provide a reliable estimate.

Conclusion

By performing importance sampling in the latent space of a Normalizing Flow, this research aligns the geometry of the failure search with the underlying assumptions of standard sampling algorithms. This approach not only improves the reliability of safety estimates for complex systems but also allows for the reuse of pre-trained generative models across different failure criteria. Future work will explore scaling this method to even higher-fidelity, "black-box" industrial simulators.

Summary updated

BibTeX
@inproceedings{kruse2025enhanced,
  author = {Liam Anthony Kruse and Alexandros E. Tzikas and Harrison Delecki and Mansur M. Arief and Mykel J. Kochenderfer},
  title = {{Enhanced Importance Sampling Through Latent Space Exploration in Normalizing Flows}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i17.33978},
}
Figure