TL;DR
This paper proves that the classical Gaussian kernel is never positive-definite on Riemannian symmetric spaces, a fundamental property required for many machine learning algorithms. The authors develop a novel harmonic-analytic framework to rigorously establish this result, offering a powerful new tool for understanding invariant kernels.
What Problem Does This Paper Address?
In machine learning, particularly in kernel-based methods, positive-definite kernels are crucial. They ensure the mathematical consistency and effectiveness of algorithms like Support Vector Machines and Gaussian Processes. While many positive-definite kernels are known for Euclidean spaces, their behavior on more complex, non-Euclidean spaces like Riemannian manifolds is less understood. Specifically, the Gaussian kernel, a ubiquitous choice, has been suspected to fail this positive-definiteness property on certain non-Euclidean spaces. This paper tackles the challenge of rigorously proving or disproving the positive-definiteness of the Gaussian kernel on a broad class of these spaces: Riemannian symmetric spaces. This is important because these spaces appear in various applications, from computer vision to quantum information.
Methods
The core of this paper lies in its development and application of a harmonic-analytic approach to study invariant kernels on Riemannian symmetric spaces. Key techniques include:
- Godement's Theorem: This is a generalization of Bochner's theorem, providing a criterion for positive-definiteness of functions on homogeneous spaces. The paper highlights the practical difficulties in applying the general form of Godement's theorem.
- L²-Godement Theorem: A specialized version of Godement's theorem that significantly simplifies the application of the criterion for square-integrable functions. This theorem relates positive-definiteness to the non-negativity of the spherical transform.
- Spherical Transform: This is a generalization of the Fourier transform to symmetric spaces, acting on H-invariant functions. The paper details the spherical transform for hyperbolic planes and spaces.
- Embedding Lemma: A crucial geometric tool that allows results proven for a smaller space (like a circle or a hyperbolic plane) to be extended to larger spaces that contain them isometrically.
The authors combine these analytical tools with specific geometric insights about symmetric spaces to derive their results.
Datasets
Not discussed in the paper.
Key Insights and Ideas
- Gaussian Kernel's Failure: The most striking insight is the definitive proof that the Gaussian kernel is never positive-definite on any Riemannian symmetric space, regardless of parameter choices, with very specific exceptions in low dimensions that are handled numerically. This resolves a long-standing question.
- Power of Harmonic Analysis: The paper demonstrates the immense power of harmonic analysis, particularly through Godement's theorem and the spherical transform, in tackling complex problems in geometric learning.
- Tractability of L²-Godement: The L²-Godement theorem is shown to be a much more practical tool than the general Godement's theorem for analyzing invariant kernels.
- Geometric Embeddings as a Bridge: The embedding lemma is a clever way to transfer results from simpler spaces to more complex ones, significantly broadening the applicability of specific findings.
Key Results
- Gaussian Kernel is Never Positive-Definite: The paper provides rigorous proofs that the Gaussian kernel is not positive-definite on compact symmetric spaces (by embedding a circle) and on non-compact symmetric spaces (by embedding hyperbolic planes).
- L²-Godement Theorem Application: The paper successfully applies the L²-Godement theorem to show that the Gaussian kernel's spherical transform oscillates between positive and negative values on hyperbolic planes, leading to its non-positive-definiteness.
- Hyperbolic Secant Kernel: The paper introduces and analyzes the "hyperbolic secant kernel," which is positive-definite on the hyperbolic plane for certain parameter ranges, offering a potential alternative to the Gaussian kernel.
- General Framework: The work lays out a general framework for analyzing invariant kernels on symmetric spaces, suggesting many avenues for future research.
Limitations and Future Work
- Numerical vs. Analytical: While the paper provides a full analytical proof for the hyperbolic space, the case of the hyperbolic plane relies on numerical evaluation for certain parameter ranges. The authors express hope for a complete analytical treatment of this case.
- General Riemannian Manifolds: The current work is restricted to Riemannian symmetric spaces. Extending these results to more general Riemannian manifolds remains an open challenge.
- Other Invariant Kernels: The paper focuses heavily on the Gaussian kernel. Future work could explore the positive-definiteness of other types of invariant kernels on symmetric spaces.
- Applications: The authors suggest that their findings could lead to new kernel-based methods for applications in areas like phylogenetic trees, which also benefit from the framework of Gelfand pairs.
Summary updated