Kernel-based GAN critics are generators' teachers that compare samples through a similarity function instead of a learned classifier. They are supposed to avoid the "saturated discriminator" problem, where a critic grows so confident that the generator gets no gradient. We argue that when the data live on a curved space, such as points on the Earth's sphere, angles on a torus, or covariance matrices, these critics saturate anyway, and the cause is plain floating-point underflow. To be mathematically valid on a curved space, the similarity function has to be the heat kernel, which measures how much heat placed at one point reaches another after diffusion time $t$. To be sharp enough to resolve fine structure such as earthquake fault lines, $t$ must be small. At small $t$, the similarity between a badly placed generated sample and every real sample rounds to exactly zero in 32-bit arithmetic, and so does the generator's gradient.
We propose a critic that never evaluates those vanishing numbers in linear space. It compares distributions through a log-domain, scale-free version of the kernel discrepancy, computed entirely from log heat kernels with log-sum-exp. The generator gradient then becomes a softmax-weighted pull along geodesics toward the data. That pull has a well-defined direction no matter how far the generator has drifted. The critic adversarially learns a nonnegative mixture over diffusion times, which keeps the kernel valid and lets the critic choose how local or global its view of the manifold is.
Core Claim
Two facts motivate the work. First, the obvious kernel is invalid: substituting geodesic distance into a Gaussian kernel is positive definite only on flat spaces, as Geodesic exponential kernels: When curvature and linearity conflict proves. The Gaussian kernel on the circle and spaces that admit isometric embeddings of the circle shows it is never positive definite on the circle for any bandwidth, and therefore fails on spheres, tori and Grassmannians. Second, the valid alternative fails numerically. On the 2-sphere, $p_t(x,y)\approx \exp(-d(x,y)^2/4t)$ falls below the smallest normal fp32 number once $d^2/4t \gtrsim 87$, since $\ln(1.2\times10^{-38})\approx -87$. Because $d\le\pi$ on the unit sphere, this happens for every $t \lesssim 0.028$, which is exactly the regime a sharp critic needs.
Our claim is that, for kernel critics, discriminator saturation on manifolds is this underflow. It can be removed by changing the representation rather than handicapping the critic. Let $\mu_P$ and $\mu_Q$ be the kernel mean embeddings of the data and generator distributions, i.e. their average similarity profiles. We train on
$$D_{\log}(P,Q) = \log\langle\mu_P,\mu_P\rangle + \log\langle\mu_Q,\mu_Q\rangle - 2\log\langle\mu_P,\mu_Q\rangle,$$
where each inner product is a double average of kernel values such as $\mathbb{E}_{x\sim P,\,y\sim Q}\,k(x,y)$, so that each logarithm is a log-sum-exp over pairwise $\log k$. By Cauchy–Schwarz, $D_{\log}\ge 0$. It is zero only if $\mu_P$ is a multiple of $\mu_Q$. For a kernel that is injective on signed measures, that forces $P=cQ$, and therefore $P=Q$ because both have unit mass. On a compact manifold the spectral kernel's constant mode makes this concrete.
This is the classical Cauchy–Schwarz divergence. Our contribution is not the formula. It is the observation that, built on intrinsic log heat kernels, this divergence is the representation in which the critic's gradient survives finite precision. For a generated sample $y$, the cross term's gradient is
$$\nabla_y \log \sum_i k(x_i,y) = \sum_i \mathrm{softmax}_i\big(\log k(x_i,y)\big)\,\nabla_y \log k(x_i,y),$$
and for small $t$, $\nabla_y \log p_t(x_i,y)\approx \mathrm{Log}_y(x_i)/2t$, where $\mathrm{Log}_y$ is the Riemannian log map, the tangent vector pointing from $y$ to $x_i$ along the geodesic. This is a pull toward the nearest real data along the manifold. It is finite and well defined even when every $k(x_i,y)$ is numerically zero. The generator self-term contributes the matching repulsion among generated samples.
The critic's kernel is $k_w(x,y)=\sum_j w_j\,p_{t_j}(x,y)$ with $w_j\ge 0$ learned adversarially to maximize $D_{\log}$. Its log is again a log-sum-exp of $\log w_j + \log p_{t_j}$. This mixture is positive definite for every $w$. In frequency terms it is a learnable low-pass filter $g(\lambda)=\sum_j w_j e^{-t_j\lambda}$ on the Laplace–Beltrami spectrum, the manifold's notion of frequency. That makes it the manifold counterpart of the learnable cutoff in Butterworth as Attention, which gates flat 2D FFT frequencies to give a global receptive field. Small-$t$ components are local "receptive fields", large-$t$ components see the whole manifold, and the critic chooses the balance.
Relation to Prior Work
The motivating evidence comes from the kernel-GAN line. MMD GAN replaces the saturating Jensen–Shannon critic with maximum mean discrepancy, a kernel distance between distributions, over a learned feature map. Improving MMD-GAN Training with Repulsive Loss Function adds a repulsive critic loss and a bounded RBF kernel to stop kernel gradients from dying. MAGAN handicaps an energy-based critic by adapting its margin. All three work in flat feature space and evaluate kernels in linear space. The bounded kernel clips after exponentiation, so it does not prevent the underflow we target.
On the Convergence of Gradient Descent in GANs treats training as a flow on the curved space of distributions, but its data space is flat. We differ from all of these by making the data space curved and the kernel intrinsic, and by fixing saturation through the arithmetic rather than through critic handicaps.
The method borrows from three lines of work. Valid manifold kernels come from Matérn Gaussian Processes on Riemannian Manifolds and Geometry-aware Bayesian Optimization in Robotics using Riemannian Matérn Kernels. These are spectral and heat-kernel constructions used only for passive regression and optimization, evaluated in linear space. Log heat kernels on general manifolds come from Riemannian Diffusion Models on General Manifolds via Physics-Informed Neural Networks, which trains a network to output $\log p_t$ for score-matching diffusion. We reuse that surrogate where no closed form exists, but put it inside an adversarial critic. Signed log-sum-exp from Subtractive Mixture Models via Squaring handles spectral sums whose eigenfunction products change sign.
One constraint should be stated up front. A signed spectral series cannot resolve values far below its largest term, because cancellation limits its accuracy. Log-space bookkeeping does not cure cancellation. We therefore use the spectral series only at large $t$, and use closed-form or short-time log representations at small $t$.
Research Program
We will build log-heat-kernel evaluators on three geometries.
The first is the flat torus $\mathbb{T}^d$. Its heat kernel is a sum of wrapped Gaussians with only positive terms, so $\log p_t$ is an exact log-sum-exp over winding numbers. This makes it a clean testbed with known ground truth. It also handles the wrap-around tension: repelled points approach each other again from the other side, and the wrapped sum accounts for that automatically.
The second is the sphere $S^2$. Here we use the Legendre series for large $t$ and a short-time expansion of $\log p_t$ for small $t$, with a crossover chosen where the two agree to fp32 precision.
The third is the space $\mathrm{SPD}(n)$ of symmetric positive definite matrices, which is non-compact. Here we use the log-heat-kernel surrogate of Ko and Lee. Because compactness no longer guarantees identifiability, this geometry is treated as an empirical extension.
On each geometry we will then do two things. First, we will train generators that output manifold points through the exponential map, using the critic $k_w$ and the objective $D_{\log}$. Second, we will measure the fraction of generated samples whose fp32 gradient is exactly zero, a direct saturation meter, and track it through training. On the analysis side, we will characterize when the softmax pull of the cross term causes nearest-mode capture, and whether the generator self-term counteracts it.
Baselines
The naive reference is an MMD critic with the geodesic Gaussian kernel $\exp(-\gamma d_g^2)$ at matched bandwidths. It is what practitioners actually use. Feragen et al. warn that it can look positive semi-definite on sparse samples, so we log Gram-matrix eigenvalues every 100 steps. Beating it means equal or better sample quality with no negative eigenvalues.
The strongest competitor on our question is an ambient-embedding MMD. It uses a Euclidean RBF kernel, in the repulsive bounded-kernel form of Improving MMD-GAN Training with Repulsive Loss Function, on $S^2\subset\mathbb{R}^3$, on $\mathbb{T}^d$ embedded as products of unit circles, and on vectorized SPD matrices, with a standard mixture of bandwidths. Unlike the geodesic Gaussian, this kernel is valid. On the sphere its chordal distance is a monotone function of geodesic distance. It is therefore the honest threat.
The strongest existing generator on our manifold data is the PINN-based Riemannian diffusion model of Ko and Lee. It reports a Frobenius distance to ground truth of 5.21 on NYC Taxi $\mathrm{SPD}(10)$, against 6.15 for its baseline (Riemannian Diffusion Models on General Manifolds via Physics-Informed Neural Networks). It also reports matching or beating Riemannian score-based models on Earth event data; no reported result found on Lacuna for that number.
The key ablation keeps the same learned heat-kernel mixture but trains on ordinary linear-domain squared MMD in fp32. This isolates the log-domain objective as the ingredient. A second ablation runs that linear version in fp64, and a third fixes the mixture weights to isolate adversarial kernel learning.
All methods share one fairness protocol. We use the same generator network, batch size, number of critic and generator steps, and wall-clock budget. Each method gets the same number of hyperparameter trials, with an equal-size search over bandwidths or diffusion times. Each configuration runs with five seeds.
Why not just…?
Why not just run the linear kernel in double precision? In fp64 the underflow threshold moves from $d^2/4t\approx 87$ to about $745$. On the unit sphere that only shifts the failure point from $t\approx0.028$ to $t\approx0.0033$. It does not remove it, and fp64 is far slower on GPUs. The fp64 ablation measures exactly where the zero-gradient fraction becomes nonzero. If fp64 matches us at every useful $t$ on our data, the contribution shrinks to a speed argument, and we will report that.
Why not just include large bandwidths in the kernel mixture, as MMD GANs already do? This is a real threat. Large-$t$ heat kernels never underflow on a compact manifold, so the gradient never vanishes. The cost is that the large-$t$ kernel approaches a constant, and its gradient is weak and blurry exactly when fine structure matters. Our fixed-weight and ambient multi-bandwidth baselines test this directly. If they reach the same mode coverage and sharpness in the same steps, the log-domain objective is unnecessary.
Why not just use the geodesic Laplacian kernel $\exp(-\lambda d_g)$, which is valid on spheres? Geodesic exponential kernels shows this kernel is positive definite on spheres and hyperbolic spaces, and it decays slowly enough to underflow late. On $S^2$ it is a serious baseline, and we include it. However, the same paper shows it fails for the affine-invariant SPD metric and for Grassmannians. It also offers no learned control over locality. Our SPD experiments are where the difference should appear.
Why not just use a diffusion model? Diffusion models already handle these manifolds, and Ko and Lee is our strongest baseline. A GAN's advantage is one-step sampling. If our method loses on quality, we will report the speed–quality trade-off rather than claim a win.
Proposed Experiments
The decisive check is a far-start test. On the 8-modes-on-a-circle benchmark of Mroueh and Nguyen (no reported numeric result found on Lacuna), we initialize the generator antipodal to all modes. We repeat this on $\mathbb{T}^2$ and $\mathbb{T}^4$ mixtures with exact ground-truth densities, where we compute KL divergence on a grid. The prediction is specific. Linear-domain heat MMD at $t\le 0.01$ should show a zero-gradient fraction near one and stall. Our method should reach all modes. Large-bandwidth baselines should move but leave mass smeared between modes.
On real data we use the Earth event datasets on $S^2$ (volcanoes, earthquakes, wildfires), scored by a held-out two-sample test with a fixed intrinsic kernel and by nearest-neighbor coverage. On NYC Taxi $\mathrm{SPD}(10)$ we report Frobenius distance under Ko and Lee's protocol, against their 5.21.
As a flat-space control, we run $D_{\log}$ with a learned feature map on CIFAR-10. This checks that the objective is not harmful where no curvature exists, against the reported MMD-rep-b FID of 16.21 (Improving MMD-GAN Training with Repulsive Loss Function).
The idea fails if either of two things happens. The first is that the ambient multi-bandwidth MMD matches us on the sphere, the torus and SPD. The second is that the zero-gradient fraction of the linear-domain ablation stays near zero at the diffusion times the critic actually learns.
References
- Geodesic exponential kernels: When curvature and linearity conflict
- The Gaussian Kernel on the Circle and Spaces that Admit Isometric Embeddings of the Circle
- Butterworth as Attention: Anisotropic Spectral Gating for Pansharpening
- MMD GAN: Towards Deeper Understanding of Moment Matching Network
- Improving MMD-GAN Training with Repulsive Loss Function
- MAGAN: Margin Adaptation for Generative Adversarial Networks.
- On the Convergence of Gradient Descent in GANs: MMD GAN As a Gradient Flow
- Matérn Gaussian Processes on Riemannian Manifolds
- Geometry-aware Bayesian Optimization in Robotics using Riemannian Matérn Kernels
- Riemannian Diffusion Models on General Manifolds via Physics-Informed Neural Networks
- Subtractive Mixture Models via Squaring: Representation and Learning
- Open Problem: Kernel methods on manifolds and metric spaces. What is the probability of a positive definite geodesic exponential kernel?