What is an inductive mean?

Frank NielsenarXiv 2024 · DOI · arXiv

Abstract

This paper explores the framework of inductive means—mathematical averages defined not by closed-form expressions, but as the limits of converging sequences of simpler iterations. By synthesizing classical scalar means with modern developments in matrix analysis and information geometry, we demonstrate how inductive sequences unify disparate definitions of the geometric mean for Symmetric Positive-Definite (SPD) matrices. We highlight the convergence properties of these sequences and their significance in non-Euclidean data processing.

Introduction

In elementary statistics, the "Pythagorean means"—Arithmetic ($A$), Geometric ($G$), and Harmonic ($H$)—form the bedrock of data aggregation. While these are trivially defined for scalar values, modern computational challenges in medical imaging (DT-MRI), computer vision, and radar signal processing require averaging complex data types such as Symmetric Positive-Definite (SPD) matrices.

Because matrices generally do not commute ($XY \neq YX$), the definition of a "geometric mean" becomes inherently ambiguous. Frank Nielsen’s work addresses this by proposing the inductive mean as a unifying framework. This approach views the mean as the result of an iterative process, providing a bridge between classical arithmetic and the sophisticated requirements of Riemannian geometry.

The Problem: Divergence in the Matrix World

For scalar numbers, the geometric mean $\sqrt{xy}$ can be reached via multiple mathematical paths. However, when applied to SPD matrices, these paths diverge into distinct mathematical objects:

  1. Inductive Sequences: Defining a mean as the limit of a double sequence (e.g., repeatedly taking the $A$ and $H$ of two numbers).
  2. Quasi-Arithmetic Means: Defining a mean via a functional generator $f(u)$, such as the Log-Euclidean mean where $f(u) = \log(u)$.
  3. Limits of Power Means: Viewing the geometric mean as the limit of the $p$-power mean as $p \to 0$.

In the matrix manifold, these paths yield different results, such as the Riemannian geometric mean versus the Log-Euclidean mean. The inductive framework allows us to understand these relationships and compute them efficiently.

Methodological Framework

The Riemannian Approach

The paper emphasizes the Riemannian perspective, where the matrix geometric mean is the midpoint of a geodesic—the shortest path—on the manifold of SPD matrices. This is often referred to as the Cartan mean or the Riemannian center of mass. It is mathematically characterized as the point that minimizes the sum of squared Riemannian distances to the input matrices.

Information Geometry

Beyond standard Euclidean space, means are linked to dual affine connections and Hessian manifolds. This perspective allows for an empirical Law of Large Numbers (LLN) on non-Euclidean spaces, where one can estimate the expectation of matrix-valued random variables by incrementally updating their Riemannian mean through inductive steps.

Key Insights and Results

Quadratic Convergence

A significant finding is the efficiency of the matrix Arithmetic-Harmonic Mean (AHM) sequence. The paper proves that this sequence converges quadratically to the matrix geometric mean: $$G(X, Y) = X^{1/2}(X^{-1/2}YX^{-1/2})^{1/2}X^{1/2}$$ This provides a numerically stable method to calculate matrix square roots and geometric means without the computational burden of repeated eigendecompositions.

Operator Monotonicity

A critical distinction identified in the research is operator monotonicity. The Riemannian geometric mean preserves the Loewner partial order (if $X \leq Y$, then the mean reflects this order), a property that the simpler Log-Euclidean mean fails to maintain. This makes the inductive/Riemannian approach superior for applications requiring physical consistency.

Scaling to $n$ Variables

While two-variable means are well-understood, the paper discusses the generalization to $n$ variables. Using Holbrook’s inductive sequences, the Kärcher mean (the $n$-variable geometric mean) can be approximated by cyclically moving toward the next matrix in a set, effectively extending the inductive principle to high-dimensional data clusters.

Limitations and Future Directions

Despite the elegance of inductive means, computational hurdles remain. While some sequences converge quadratically, others, like the Ando-Li-Mathias (ALM) mean for $n$ matrices, converge only linearly, which can be prohibitive for large-scale datasets. Furthermore, in information geometry, multivariate functions are often only locally invertible, complicating the generalization of quasi-arithmetic means.

Future research is directed toward generalizing these iterations to self-dual operators on convex functionals within Hilbert spaces. This could pave the way for new "convex geometric means" tailored for functional data analysis and complex machine learning architectures.

Conclusion

Inductive means offer a powerful, iterative lens through which to view the concept of an "average." By moving away from static formulas and toward convergent processes, we gain the tools necessary to navigate the non-commutative and non-Euclidean geometries of modern data science.

Summary updated

Abstract

An inductive mean is a mean defined as a limit of a convergence sequence of other means. Historically, this notion of inductive means obtained as limits of sequences was pioneered independently by Lagrange and Gauss for defining the arithmetic-geometric mean. In this note, we first explain several generalizations of the scalar geometric mean to symmetric positive-definite matrices, and then present several inductive mean mechanisms for sets of symmetric positive-definite matrices.
BibTeX
@misc{nielsen2024what,
  author = {Frank Nielsen},
  title = {{What is an inductive mean?}},
  year = {2024},
  eprint = {2410.15717},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.15717},
}
Figure