ExGAN: Adversarial Generation of Extreme Samples

Siddharth Bhatia, Arjit Jain, Bryan Hooi

Abstract

Generative models typically struggle with "extreme" samples—rare events in the long tails of a distribution, such as catastrophic floods or financial crashes. ExGAN is a novel framework that combines Generative Adversarial Networks (GANs) with Extreme Value Theory (EVT) to generate realistic, highly improbable scenarios in constant time. By addressing data scarcity through iterative distribution shifting and ensuring physical realism via EVT-based conditional generation, ExGAN overcomes the efficiency bottlenecks of traditional rejection sampling.

Introduction

Standard deep generative models excel at capturing the "bulk" of a data distribution, producing typical samples like standard human faces or common weather patterns. However, in safety-critical domains such as disaster management and infrastructure planning, the primary concern lies in the extremes. Generating these rare events is challenging due to data scarcity—extreme events are rare by definition—and sampling inefficiency. Traditional methods like rejection sampling require generating thousands of typical samples to find one extreme event, leading to exponential time complexity as rarity increases.

Methodology

ExGAN introduces two primary innovations to bridge the gap between deep learning and the mathematics of rare events.

Distribution Shifting

To combat data scarcity, the authors propose an iterative Distribution Shifting approach. Rather than training on the entire dataset simultaneously, the model gradually shifts its focus toward the tail. In each iteration, the least extreme samples are filtered out and replaced with generated samples that are incrementally more extreme. This "iterative zoom" uses warm starts to maintain GAN stability while navigating deeper into the distribution's tail.

Spatial precipitation patterns captured by ExGAN.

EVT-based Conditional Generation

The framework leverages Extreme Value Theory (EVT), specifically the Generalized Pareto Distribution (GPD), to model tails mathematically. A Conditional GAN (CGAN) is trained where the "extremeness" (e.g., total rainfall) is an input variable. A specific loss term, $L_{ext}$, penalizes the model if the generated sample's actual extremeness deviates from the requested input. This allows users to specify a target probability (e.g., a 1-in-1,000-year event) and receive a realistic sample immediately.

Experimental Results

The model was evaluated using the US Precipitation Dataset, featuring daily spatial rainfall distributions.

Efficiency and Quality

ExGAN generates extreme samples in constant time (~0.002s), regardless of the target rarity. For an extremeness probability of $\tau = 0.01$, ExGAN is approximately 3,700x faster than a standard DCGAN using rejection sampling. At $\tau = 0.001$, standard models often fail to produce a single valid sample within an hour, while ExGAN remains efficient.

Quantitatively, ExGAN achieved a lower Fréchet Inception Distance (FID) of 0.0236 (compared to 0.0406 for DCGAN) and lower reconstruction loss, indicating higher realism in the generated "floods."

Visual Fidelity

Qualitative analysis confirms that ExGAN learns complex spatial features. Unlike simple statistical models that might uniformly increase rainfall, ExGAN recognizes that extreme precipitation follows specific geographic patterns, such as concentration in the Southeast United States.

Generated extreme rainfall events showing high-intensity patterns.

Conclusion and Future Work

ExGAN successfully breaks the rarity-time tradeoff, providing a curriculum for GANs to learn rare event features without collapsing. While the framework is principled, it currently lacks the rigorous theoretical guarantees of pure statistical EVT models. Future research may involve learning extremeness measures directly from data via neural networks or exploring Bayesian architectures for improved uncertainty estimation in high-stakes scenarios.

Summary updated

BibTeX
@inproceedings{bhatia2021exgan,
  author = {Siddharth Bhatia and Arjit Jain and Bryan Hooi},
  title = {{ExGAN: Adversarial Generation of Extreme Samples}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i8.16834},
}
Figure