Abstract
We extend the classical Mercer theorem to reproducing kernel Hilbert spaces (RKHS) whose elements are functions from a measurable space $X$ into $\mathbb{C}^n$. Given a finite measure $\mu$ on $X$, we represent the reproducing kernel $K$ as a convergent series in terms of the eigenfunctions of a suitable compact operator depending on $K$ and $\mu$. Our result holds under the mild assumption that $K$ is measurable and the associated Hilbert space is separable. Furthermore, we show that $X$ has a natural second countable topology with respect to which the eigenfunctions are continuous and the series representing $K$ converges uniformly on any compact subsets of $X \times X$, provided that the support of $\mu$ is $X$.
Introduction
In the realm of machine learning and functional analysis, Mercer’s theorem serves as a fundamental bridge between kernel functions and feature spaces. It provides the theoretical justification for the "kernel trick," allowing complex nonlinear patterns to be represented as inner products in high-dimensional Hilbert spaces. Traditionally, however, Mercer’s theorem has been constrained by strict topological requirements: the input space $X$ must be a compact metric space and the kernel $K$ must be continuous.
Modern data science frequently encounters scenarios that violate these assumptions. Inputs may be abstract entities like graphs, strings, or manifolds without a pre-defined metric. Furthermore, multi-task learning and structured output prediction often require vector-valued kernels—functions that output matrices or operators rather than scalars. This paper addresses these gaps by generalizing Mercer’s theorem to vector-valued kernels on general measurable spaces, requiring only minimal assumptions of measurability and separability.
Theoretical Framework
The Induced Topology
The core innovation of this work is the shift away from an external topology. Instead of requiring $X$ to be a metric space, the authors use the kernel $K$ itself to induce a "natural" pseudo-metric on $X$:
$$d(x, t) = \sup_{\|y\| \le 1} \|K_x y - K_t y\|_K$$
This intrinsic metric defines a topology (denoted as $\tau_K$) that makes the kernel continuous by construction. This approach ensures that the geometry of the space is perfectly aligned with the information captured by the kernel. The authors demonstrate that this induced topology is second countable, ensuring it remains mathematically tractable.
Operator Theory and Measure Weighting
To derive the eigen-decomposition, the authors analyze the canonical embedding from the RKHS into the $L^2$ space. A significant technical challenge arises when the kernel is not bounded in the traditional sense. To resolve this, the authors introduce a reweighted measure $\nu$, derived from the original measure $\mu$ and the trace of the kernel. This transformation ensures that the resulting integral operator is trace-class, which is a necessary condition for a well-behaved compact operator decomposition.
The Main Result: Vector-Valued Mercer Representation
The primary contribution of the paper is Theorem 3.2, which provides a strong version of Mercer’s theorem for the vector-valued setting. For a measurable, separable kernel $K$ taking values in $\mathbb{C}^n$, there exists a countable family of continuous eigenfunctions $\{f_i\}$ and non-negative eigenvalues $\{\sigma_i\}$ such that:
$$K(x, t) = \sum_{i=1}^\infty \sigma_i f_i(t) f_i(x)^*$$
Convergence Properties
Unlike previous generalizations that only guaranteed convergence "almost everywhere" (ignoring sets of measure zero), this result establishes:
- Absolute and Pointwise Convergence: The series converges for all $x, t$ within the support of the measure $\mu$.
- Uniform Convergence: The series converges uniformly on any compact subset of the support of $\mu$ relative to the induced topology $\tau_K$.
This ensures that the Mercer representation is not just a theoretical abstraction but a reliable pointwise approximation for practical use.
Implications for Kernel Methods
This generalization has significant implications for how we understand feature maps. The set of functions $\{\sqrt{\sigma_i} f_i\}$ forms an orthonormal basis for the RKHS, providing a canonical feature map for vector-valued data. By proving that this representation holds even on abstract measurable spaces, the authors provide a rigorous foundation for applying kernel methods to non-Euclidean data types, such as categorical data or complex structured objects, provided a measurable kernel can be defined.
Limitations and Future Directions
While the paper focuses on outputs in $\mathbb{C}^n$ for clarity, the authors suggest that the framework can be extended to infinite-dimensional separable Hilbert spaces, provided the kernel operator $K(x, x)$ remains compact. However, the specific nuances of infinite-dimensional outputs remain a subject for further study.
Additionally, the convergence is strictly tied to the support of the measure $\mu$. If the data distribution does not cover certain regions of the input space $X$, the Mercer representation may not be valid in those "unseen" areas. Finally, as a theoretical work, the paper does not provide algorithms for the numerical computation of these eigenfunctions, which remains a practical hurdle for large-scale applications.
Summary updated