Rare Event Detection Using Disentangled Representation Learning

Ryuhei Hamaguchi, Ken Sakurada, Ryosuke NakamuraCVPR 2019 · DOI

Abstract

Detecting rare events in high-dimensional data, such as building changes in satellite imagery, is challenging due to extreme class imbalance and the presence of "trivial" noise like lighting and seasonal variations. This paper proposes a novel framework for rare event detection using disentangled representation learning. By employing a dual-branch Variational Auto-encoder (VAE) trained primarily on abundant negative samples, the model learns to separate invariant scene content ("common features") from variant noise ("specific features"). This disentanglement allows for robust detection even when positive training labels are extremely scarce.

Introduction

In many real-world computer vision tasks, the target event is a "needle in a haystack." For instance, in satellite monitoring, a new building construction is a rare event, whereas changes in shadows, seasons, or illumination are trivial events that occur constantly. Two major hurdles impede effective detection: severe class imbalance, where positive samples are labor-intensive to label, and trivial noise, which often leads to high false-alarm rates as standard models fail to distinguish between normal environmental changes and significant rare events.

Methods

The core of the proposed approach is a novel architecture for Disentangled Representation Learning. The objective is to encode an image pair into two distinct latent spaces:

  1. Common Features ($z^c$): Representing the invariant image content (the "what" of the scene).
  2. Specific Features ($z^s$): Representing variant factors such as lighting, registration errors, or background motion (the "noise").
The dual-branch VAE architecture for disentangled representation learning, showing the separation into specific and common latent spaces and the associated loss functions.

The model utilizes a dual-branch VAE with shared parameters. It is trained to reconstruct input images while enforcing separation in the latent space through several key loss functions:

  • Similarity Loss ($L_{sim}$): This forces the "common" features of two images in a negative pair (where no rare event occurs) to be similar. It employs a Mahalanobis-style distance to account for the variance across different latent dimensions.
  • Activation Loss ($L_{act}$): This prevents the model from collapsing by ensuring that common features remain active and informative rather than being set to zero.

Training Strategy

The authors implement a two-phase training pipeline designed to leverage the abundance of negative samples.

The two-phase training process: (left) representation learning on negative samples to disentangle features, and (right) fine-tuning a classifier on the common features using limited positive/negative samples.
  1. Representation Learning: The VAEs are trained on a large set of negative image pairs. Since these pairs contain only trivial changes, the model learns to identify what "usually" changes (noise) versus what stays the same (content).
  2. Fine-tuning: A small classifier is trained on top of the learned "common" features using the few available positive samples. By stripping away the "specific" noise features, the downstream classification task is significantly simplified.

Datasets and Evaluation

The method was evaluated across four diverse datasets:

  • Augmented MNIST: A synthetic benchmark with added rotation and clutter.
  • ABCD Dataset: Aerial imagery for post-earthquake building change detection.
  • PCD Dataset: Street-view panoramic images.
  • WDC Dataset: A new large-scale aerial dataset of Washington D.C. (1995-2015) for building construction/destruction detection.

Key Results and Insights

The proposed method consistently outperformed baselines such as standard Siamese networks and transfer learning, particularly in scenarios of extreme label scarcity.

  • Extreme Scarcity: On Augmented MNIST with only 50 positive labels, the method achieved ~80% accuracy, whereas standard baselines performed near random guess levels (~50%).
  • Real-World Performance: On the ABCD dataset with only 5 positive labels, the method reached 78.5% accuracy, compared to 61% for standard under-sampling.

A crucial insight is that negative samples are sufficient to learn a robust representation of "normalcy." By learning the distribution of trivial changes, the model implicitly learns to isolate the invariant content. Furthermore, the use of Mahalanobis distance over simple Euclidean distance proved vital, as it correctly scales the similarity measure based on the variance of latent dimensions.

Conclusion

This work demonstrates that disentangled representation learning is a powerful tool for rare event detection. By treating disentanglement as a denoising filter, the model focuses on the core content of a scene, making it highly effective even with minimal supervision. Future work may explore extending this framework to pixel-level localization and incorporating adversarial training to further refine the separation between content and noise.

Summary updated

BibTeX
@inproceedings{hamaguchi2019rare,
  author = {Ryuhei Hamaguchi and Ken Sakurada and Ryosuke Nakamura},
  title = {{Rare Event Detection Using Disentangled Representation Learning}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2019},
  doi = {10.1109/cvpr.2019.00955},
}
Figure