Fréchet Sufficient Dimension Reduction for Metric Space-Valued Data via Distance Covariance

Hsin-Hsiung Huang, Feng Yu, Kang Li, Teng ZhangJournal of Computational and Graphical Statistics 2025 · DOI

TL;DR

This paper introduces a novel method called Fréchet Sufficient Dimension Reduction via Kernel Distance Covariance (Fd-SDR) that extends dimension reduction techniques to complex, non-Euclidean data like probability densities and gene expression profiles. By leveraging kernel distance covariance, Fd-SDR offers a more flexible and computationally efficient approach to uncovering essential relationships in high-dimensional data compared to existing methods.

What Problem Does This Paper Address?

Traditional statistical methods often struggle with "metric space-valued data," which includes complex objects like probability distributions, density functions, or even images, rather than simple numbers or vectors. These data types are increasingly common in fields like bioinformatics, environmental science, and economics. Standard dimension reduction techniques, designed for simpler data, fail to capture the intricate geometric properties of these complex data structures. This "curse of dimensionality" makes it difficult to analyze and interpret relationships between these complex responses and high-dimensional predictors. Existing Fréchet SDR methods, while designed for metric spaces, face computational challenges, especially with very high-dimensional predictor spaces.

Methods

The core innovation is the development of Fréchet Sufficient Dimension Reduction via Kernel Distance Covariance (Fd-SDR). This method builds upon the concept of Fréchet regression, which is designed for metric space-valued responses. The key steps are:

  • Kernel-based Feature Mapping: Metric space-valued responses are transformed into a higher-dimensional feature space using a kernel function. This allows standard distance-based measures to be applied.
  • Kernel Distance Covariance (KdCov): Instead of traditional distance covariance, the paper uses KdCov to measure the dependency between the (transformed) response and the predictors. KdCov is specifically designed to handle metric spaces and can capture complex, non-linear relationships.
  • Optimization on Stiefel Manifold: The dimension reduction problem is framed as an optimization problem on the Stiefel manifold, which allows for efficient computation and avoids the need for matrix inversions that are problematic in high-dimensional settings.
  • Algorithm: A specific algorithm (Algorithm 1) is proposed, detailing the steps for computing the kernel matrix, performing optimization, and obtaining the estimated coefficient matrix.

Datasets

The paper evaluates Fd-SDR on four real-world datasets:

  • Global Human Mortality Data: Age-at-death distributions for 162 countries, treated as smoothed probability density functions.
  • Washington D.C. Bike Rental Data: Hourly and daily counts of bike rentals, along with weather and temporal features.
  • Carcinomas Data: Gene expression profiles from 174 samples across 11 carcinoma types.
  • TCGA Breast Cancer Survival Data: Gene expression data, survival endpoints, and clinical data from 1,218 breast cancer patients.

The authors also use synthetic datasets generated under various models to evaluate performance.

Key Insights and Ideas

  • Bridging Metric Spaces and SDR: The paper successfully bridges the gap between complex metric space-valued data and the powerful framework of Sufficient Dimension Reduction, opening up new avenues for analyzing previously intractable data.
  • KdCov for Non-Linearity: The use of Kernel Distance Covariance is crucial for capturing non-linear dependencies, which are often missed by traditional SDR methods that focus on linear relationships.
  • Computational Efficiency: The proposed method is significantly faster than existing Fréchet SDR approaches, making it practical for high-dimensional and large-scale datasets. This is achieved through clever optimization techniques that avoid computationally expensive matrix operations.
  • Broad Applicability: The method's theoretical guarantees and empirical performance across diverse datasets (distributions, counts, gene expression, survival data) highlight its broad applicability.

Key Results

  • Superior Accuracy: In synthetic simulations, Fd-SDR consistently achieved smaller estimation errors compared to FOPG and GWIRE, especially in challenging scenarios where other methods failed.
  • Significant Speedup: Fd-SDR demonstrated substantial computational advantages, being 10-20 times faster than FOPG and 50-100 times faster than GWIRE in running time comparisons.
  • Real-World Performance: On real-world datasets, Fd-SDR achieved competitive or superior performance in terms of R-squared values (mortality, bike rental, breast cancer survival) and AUC scores (carcinomas data). For instance, it achieved the highest AUC for carcinoma classification.

Limitations and Future Work

  • Kernel and Distance Choice: While the paper provides guidance on kernel and distance selection, the performance of Fd-SDR can still depend on these choices. The paper mentions that the choice of distance is data-dependent (e.g., Wasserstein for distributions, Frobenius for matrices).
  • CMS-Ensemble Assumption: The theoretical guarantees rely on the assumption that the family of functions induced by the kernel forms a CMS-ensemble, which is satisfied by common kernels like Gaussian and Laplacian.
  • Future Work: The paper suggests that the framework can be extended to other separable and complete metric spaces of negative type, indicating potential for further applications. The authors also highlight the importance of the CMS framework for identifying sufficient predictors, suggesting continued exploration in this area.

Summary updated

BibTeX
@article{huang2025frechet,
  author = {Hsin-Hsiung Huang and Feng Yu and Kang Li and Teng Zhang},
  title = {{Fréchet Sufficient Dimension Reduction for Metric Space-Valued Data via Distance Covariance}},
  journal = {Journal of Computational and Graphical Statistics},
  year = {2025},
  doi = {10.1080/10618600.2025.2602521},
}
Figure