In a standard Generative Adversarial Network (GAN), the discriminator acts as a binary judge. It looks at a single image and asks: "Is this real or fake?" While this setup has produced impressive results, it suffers from a fundamental logical flaw that often leads to training instability and "mode collapse."
Alexia Jolicoeur-Martineau identified this gap in The relativistic discriminator: a key element missing from standard GAN: in a standard GAN, as the generator improves and produces more realistic images, the discriminator’s probability for real data does not necessarily decrease. Logically, if the generator is getting better, the discriminator should become less certain that the real data is the "most" realistic thing it has seen.
Relativistic GANs (RGANs) solve this by shifting the discriminator's objective from absolute classification to relative ranking.
The Logical Gap in Absolute Classification
In the standard GAN framework, the discriminator $D(x)$ estimates the probability that an input $x$ is real. The generator $G$ tries to increase $D(G(z))$ while the discriminator tries to decrease it.
The problem is that the discriminator evaluates samples in isolation. If the generator produces a perfect replica of the training data, a standard discriminator might still output "0.5" for both real and fake images, but the gradients often vanish before reaching that point. Because the discriminator is not required to compare the two distributions, it can become "too perfect" too quickly, saturating its activation function and providing no useful feedback (gradients) to the generator.
Shifting to Relative Realism
The core intuition of the Relativistic Discriminator is that it should estimate the probability that a real sample $x_r$ is more realistic than a randomly sampled fake sample $x_f$.
Instead of outputting a score for one image, the relativistic objective compares pairs: $$D(x_r, x_f) = \text{sigmoid}(C(x_r) - C(x_f))$$ where $C(x)$ is the non-activated output (critic score) of the network.
This change ensures that the generator receives a continuous gradient signal. To improve, the generator doesn't just need to make "realistic" images; it needs to make images that are more realistic than the ones it made in the previous iteration, relative to the real data. As shown in On Relativistic f-Divergences, this approach provides a more principled mathematical foundation for adversarial learning, ensuring that the discriminator's certainty in real data decreases as the generator's quality increases.
The Relativistic Average (RaGAN)
Comparing individual pairs of images can be noisy. To stabilize this, researchers often use the Relativistic Average Discriminator (RaD). Instead of comparing one real image to one fake image, the discriminator compares a real image to the average realism of a batch of fake images.
As Alexia Jolicoeur-Martineau explains, this averaging mechanism reduces variance and prevents the discriminator from becoming too dominant early in training. This stability is crucial for high-resolution tasks. For example, PathologyGAN uses a relativistic average discriminator to capture the intricate morphological details of cancer tissue, which would otherwise be lost to training oscillations.
Recovering High-Frequency Textures
One of the most significant impacts of relativistic training is the synthesis of fine textures. Standard GANs often produce "cartoonish" or overly smooth results because the discriminator saturates before it can force the generator to learn micro-details.
In the landmark paper ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks, Xintao Wang and colleagues replaced the standard discriminator with a relativistic one. This allowed the model to hallucinate sharper edges and more realistic high-frequency textures in upscaled images. By focusing on the relative difference in realism, the generator is pushed to recover the "crispness" of real-world gradients that are typically lost in super-resolution.
This same principle has been applied across diverse domains:
- Medical Imaging: Tsung-Wei Pan used relativistic pairing in R3GAN to generate high-quality embryonic images from very small datasets, where standard GANs would typically fail.
- Biometrics: Cunjian Chen utilized RaSGAN to synthesize iris images with enough textural fidelity to test security systems against presentation attacks.
- Speech Enhancement: Feng Dang and Guochen Yu applied relativistic objectives to speech enhancement, helping generators recover clean audio signals from noisy, unpaired data.
A Modern Baseline
While diffusion models have recently dominated the generative landscape, the relativistic approach has kept GANs competitive. Nick Huang and Aaron Gokaslan recently introduced R3GAN, a "modern GAN baseline" that uses regularized relativistic loss. By stripping away the complex "bag of tricks" usually required to train GANs and relying on the principled stability of relativistic comparisons, R3GAN achieves performance that rivals state-of-the-art diffusion models while remaining significantly faster to sample.