A Non-Adversarial Approach to Idempotent Generative Modelling

Mohammed Al-Jaff, Giovanni Luca Marchetti, Michael C. Welle, Jens Lundell, Mats G. Gustafsson, Gustav Eje Henter, Hossein Azizpour, Danica KragicFrontiers in Artificial Intelligence and Applications 2025 · DOI · OpenAlex

Abstract

Idempotent Generative Networks (IGNs) are deep generative models that also function as local data manifold projectors, mapping arbitrary inputs back onto the manifold. They are trained to act as identity operators on the data and as idempotent operators off the data manifold. However, IGNs suffer from mode collapse, mode dropping, and training instability due to their objectives, which contain adversarial components and can cause the model to cover the data manifold only partially – an issue shared with generative adversarial networks. We introduce Non-Adversarial Idempotent Generative Networks (NAIGNs) to address these issues. Our loss function combines reconstruction with the non-adversarial generative objective of Implicit Maximum Likelihood Estimation (IMLE). This improves on IGN’s ability to restore corrupted data and generate new samples that closely match the data distribution. We moreover demonstrate that NAIGNs implicitly learn the distance field to the data manifold, as well as an energy-based model.

Introduction

In generative modeling, researchers typically seek two primary capabilities: the ability to create new, realistic samples from noise (generation) and the ability to "fix" corrupted data by pushing it back toward the valid data distribution (manifold projection).

Idempotent Generative Networks (IGNs) were recently proposed to unify these tasks. Their defining trait is idempotency: the mathematical property where applying a function twice yields the same result as applying it once ($f(f(z)) = f(z)$). This ensures that once a point is projected onto the data manifold, it remains a fixed point. However, original IGNs rely on an adversarial objective—a "tightness" loss that pushes outputs away from inputs to prevent the model from learning a trivial identity function. Like GANs, this makes them difficult to train and prone to mode collapse and mode dropping.

The NAIGN Framework

We introduce NAIGNs, which replace the unstable adversarial "tightness" loss with a stable, non-adversarial approach based on Implicit Maximum Likelihood Estimation (IMLE). The core of the method relies on three integrated loss components:

  1. Reconstruction Loss ($L_{rec}$): Ensures that if a point is already on the data manifold, the model acts as an identity operator ($f(x) = x$).
  2. IMLE Loss ($L_{IMLE}$): This serves as the generative engine. IMLE ensures that for every real data point in a batch, there is at least one generated sample nearby. This explicitly forces the model to cover the entire data distribution.
  3. Modified Idempotency Loss ($L_{idem}$): Encourages the model to treat its own outputs as fixed points, ensuring that $f(f(z)) \approx f(z)$.
Conceptual diagram of the NAIGN framework, illustrating the roles of reconstruction, IMLE, and idempotency losses in mapping points to the data manifold M.

By combining these, the model learns to map the entire ambient space onto the data manifold while ensuring the manifold itself is a fixed point.

Key Insights and Results

Stability and Mode Coverage

Unlike GANs, where a generator and discriminator play a zero-sum game, IMLE provides a stable target. In synthetic benchmarks such as "grids" and "8gaussians," the original IGN often collapsed to just a few points or missed the outer edges of the distribution. In contrast, NAIGN successfully covered all modes.

Comparison of mode coverage on 2D synthetic datasets (Grids, 2Moons, 8Gaussians) between the original IGN (left) and NAIGN (right).

On MNIST, the original IGN's performance diverged sharply after 40,000 iterations, while NAIGN's performance improved smoothly and remained stable throughout training. This stability was further evidenced on the FFHQ-100 dataset, where NAIGN produced diverse, recognizable faces under extreme data scarcity, whereas the adversarial IGN suffered from complete mode collapse.

Implicit Distance Fields and EBMs

A striking finding is that NAIGNs implicitly learn a Manifold Distance Field. The "drift"—the distance an input moves when processed by the model—serves as a proxy for how far that input is from the true data distribution. Because the model learns this distance field, it can function as an Energy-Based Model (EBM), which is useful for tasks like out-of-distribution detection without requiring a separate model.

Conclusion and Future Work

NAIGNs provide a robust, non-adversarial alternative for idempotent generative modeling, solving the stability and mode-coverage issues inherent in original IGNs. While currently limited by pixel-space distances, future work may involve moving to latent spaces (similar to Diffusion Models) to handle higher-resolution semantic data more efficiently. Additionally, the "minimal drift" approach shows promise for learning and generating 3D shapes via Implicit Neural Representations.

Summary updated

Abstract

Idempotent Generative Networks (IGNs) are deep generative models that also function as local data manifold projectors, mapping arbitrary inputs back onto the manifold. They are trained to act as identity operators on the data and as idempotent operators off the data manifold. However, IGNs suffer from mode collapse, mode dropping, and training instability due to their objectives, which contain adversarial components and can cause the model to cover the data manifold only partially – an issue shared with generative adversarial networks. We introduce Non-Adversarial Idempotent Generative Networks (NAIGNs) to address these issues. Our loss function combines reconstruction with the non-adversarial generative objective of Implicit Maximum Likelihood Estimation (IMLE). This improves on IGN’s ability to restore corrupted data and generate new samples that closely match the data distribution. We moreover demonstrate that NAIGNs implicitly learn the distance field to the data manifold, as well as an energy-based model.
BibTeX
@inproceedings{aljaff2025non,
  author = {Mohammed Al-Jaff and Giovanni Luca Marchetti and Michael C. Welle and Jens Lundell and Mats G. Gustafsson and Gustav Eje Henter and Hossein Azizpour and Danica Kragic},
  title = {{A Non-Adversarial Approach to Idempotent Generative Modelling}},
  booktitle = {Frontiers in Artificial Intelligence and Applications},
  year = {2025},
  doi = {10.3233/faia251035},
}
Figure