Geometric Enclosing Networks

Trung Le, Hung Vu, Tu Dinh Nguyen, Dinh Q. Phung

Abstract

Training models to generate data has increasingly attracted research attention and become important in modern world applications. We propose in this paper a new geometry-based optimization approach to address this problem. Orthogonal to current state-of-the-art density-based approaches, most notably VAE and GAN, we present a fresh new idea that borrows the principle of minimal enclosing ball to train a generator $G(\mathbf{z})$ in such a way that both training and generated data, after being mapped to the feature space, are enclosed in the same sphere. We develop theory to guarantee that the mapping is bijective so that its inverse from feature space to data space results in expressive nonlinear contours to describe the data manifold, hence ensuring data generated are also lying on the data manifold learned from training data. Our model enjoys a nice geometric interpretation, hence termed Geometric Enclosing Networks (GEN), and possesses some key advantages over its rivals, namely simple and easy-to-control optimization formulation, avoidance of mode collapsing and efficiently learn data manifold representation in a completely unsupervised manner. We conducted extensive experiments on synthesis and real-world datasets to illustrate the behaviors, strength and weakness of our proposed GEN, in particular its ability to handle multi-modal data and quality of generated data.

Introduction

Generative models like Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs) have dominated the field by framing data generation as a density estimation problem—trying to match the probability distribution of generated data to the true, unknown data distribution.

However, training GANs is notoriously finicky. They frequently suffer from convergence issues and "mode collapse," a failure state where the generator produces only a limited variety of outputs (e.g., generating only one type of face), completely ignoring other parts of the data distribution. Geometric Enclosing Networks (GEN) seeks to abandon the density estimation paradigm entirely, tackling data generation through a purely geometric lens.

Methodology

GEN relies on the concept of a Minimal Enclosing Ball (MEB) from support vector machine theory. The training process is split into two distinct phases:

  1. Learning the Boundary: The input data is mapped into a finite-dimensional feature space using a Fourier random feature representation. The network then learns the center and radius of a minimal enclosing ball in this feature space that encapsulates the mapped training data.
  2. Training the Generator: A neural network generator takes random noise ($z$) and is trained via backpropagation to ensure that its generated outputs, when mapped to the feature space, fall strictly inside the learned ball.
Conceptual overview of the Geometric Enclosing Network (GEN) architecture, showing the mapping from input space to a feature space enclosed by a minimal ball.

To prevent the generator from lazily mapping all noise to the exact center of the ball, an augmented loss term encourages generated samples to spread out across the ball's volume. By framing generation as finding a geometric boundary rather than matching a probability distribution, the optimization landscape becomes simpler and more stable.

Experimental Results

The authors evaluated GEN on synthetic datasets (mixtures of Gaussians and S-shaped manifolds) and real-world image datasets (MNIST, CIFAR-10, CelebA).

On synthetic data, GEN exhibited highly stable training dynamics. In 1D and 2D Gaussian mixture tasks, GEN successfully generated data across all distinct modes. In contrast, standard GAN baselines suffered from severe mode collapse, often getting stuck on a single cluster. Quantitatively, GEN achieved a Wasserstein distance to the true distribution of nearly zero on these tasks.

Comparison of 1D generated distributions: GEN successfully captures all three modes of the data distribution, whereas the GAN baseline suffers from mode collapse.

For real-world images, GEN successfully generated visually coherent images for MNIST and Frey Face, even with limited training data. While the model generated recognizable shapes and colors for complex datasets like CIFAR-10 and CelebA, the outputs were notably lower in contrast and blurrier than those from state-of-the-art GANs.

Discussion and Conclusion

GEN offers an inherent resistance to mode collapse because the generator is forced to populate the volume of the enclosing ball, which represents the full data manifold. Furthermore, the authors mathematically prove that under specific conditions, their random feature mapping is a bijection. This guarantees that the simple spherical boundary in the feature space translates back into complex, expressive, and valid nonlinear contours in the original pixel space.

The primary limitation of GEN is currently visual fidelity on high-dimensional color images. The reliance on random Fourier features may struggle to capture the ultra-fine details required for photorealism. However, as a foundational step, GEN provides a stable, orthogonal approach to generative modeling that avoids the adversarial instability of GANs.

Summary updated

Abstract

Training model to generate data has increasingly attracted research attention and become important in modern world applications. We propose in this paper a new geometry-based optimization approach to address this problem. Orthogonal to current state-of-the-art density-based approaches, most notably VAE and GAN, we present a fresh new idea that borrows the principle of minimal enclosing ball to train a generator G\left(\bz\right) in such a way that both training and generated data, after being mapped to the feature space, are enclosed in the same sphere. We develop theory to guarantee that the mapping is bijective so that its inverse from feature space to data space results in expressive nonlinear contours to describe the data manifold, hence ensuring data generated are also lying on the data manifold learned from training data. Our model enjoys a nice geometric interpretation, hence termed Geometric Enclosing Networks (GEN), and possesses some key advantages over its rivals, namely simple and easy-to-control optimization formulation, avoidance of mode collapsing and efficiently learn data manifold representation in a completely unsupervised manner. We conducted extensive experiments on synthesis and real-world datasets to illustrate the behaviors, strength and weakness of our proposed GEN, in particular its ability to handle multi-modal data and quality of generated data.
BibTeX
@inproceedings{le2018geometric,
  author = {Trung Le and Hung Vu and Tu Dinh Nguyen and Dinh Q. Phung},
  title = {{Geometric Enclosing Networks}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2018},
  url = {https://openreview.net/forum?id=BkEyfNzuWS},
}
Figure