Tensor Decomposition with Unaligned Observations

Runshi Tang, Tamara G. Kolda, Anru R. ZhangarXiv 2024 · DOI · arXiv

Abstract

Standard tensor decomposition methods assume data is structured in a rigid grid, which is rarely the case for longitudinal studies with irregular sampling. This paper presents a framework for tensor decomposition on unaligned observations by modeling temporal modes as functions in a Reproducing Kernel Hilbert Space (RKHS). By integrating generalized loss functions and computational techniques like sketching, the proposed method enables robust analysis of non-Gaussian data (e.g., microbiome counts) with significant efficiency gains over traditional binning or interpolation methods.

Introduction: The Challenge of Unaligned Data

Standard tensor decompositions, such as Canonical Polyadic (CP), typically assume "tabular" data—a perfect multi-dimensional grid. In longitudinal studies, this would ideally be a subject-by-feature-by-time tensor. However, real-world clinical data is rarely so tidy. Subjects often visit clinics at different, irregular intervals, leading to "unaligned" observations where the time index is not consistent across the cohort.

Comparison of tabular, functional, and unaligned tensor structures.

Common workarounds like "binning" (averaging data into discrete time buckets) or interpolation often discard fine-grained information and introduce bias. Furthermore, these methods struggle with non-Gaussian data types, such as binary outcomes or sparse counts. This paper introduces a framework to perform tensor decomposition directly on these unaligned, multi-type observations without requiring pre-processing hacks.

Methodology: The RKHS-TD Framework

The authors propose treating the unaligned mode (typically time) as a continuous function rather than a discrete index. This is achieved through two primary innovations:

  1. RKHS Representation: The temporal component is modeled using functions from a Reproducing Kernel Hilbert Space (RKHS). This allows the model to represent the underlying signal as a smooth, continuous function $\xi(t)$, which can be evaluated at any arbitrary time point where an observation exists.
  2. Generalized Loss Functions: Borrowing from Generalized CP (GCP) decomposition, the framework supports various loss functions beyond squared error, such as Bernoulli loss for binary data and Poisson loss for counts.
Decomposition of unaligned observations into functional components.

Optimization and Scaling

To make the functional space computationally tractable, the authors apply the Representer Theorem, proving that the optimal solution can be represented as a finite linear combination of kernel functions at observed time points. To handle high dimensionality, they introduce Sketching (randomly sampling observations) and Stochastic Gradient Descent (SGD), which drastically reduce the computational "flops" required per iteration.

Key Insights

  • Joint Normalization: The authors found that separately normalizing subject, feature, and time factors led to numerical instability. They propose a joint normalization approach that balances the magnitudes of all factors simultaneously, significantly improving convergence.
  • Information Integrity: Unlike binning, sketching maintains the integrity of irregular time points while sampling them intelligently to speed up optimization.

Experimental Results

Synthetic Validation

In simulations, the sketched algorithm (S-RKHS-TD) demonstrated massive speedups, completing 10 iterations and reaching a lower error rate in the time it took the standard version to complete a single iteration.

Microbiome Analysis (ECAM Study)

The framework was applied to the Early Childhood Antibiotics and the Microbiome (ECAM) study, which tracks irregularly timed microbiome counts in infants. The proposed method achieved higher Silhouette scores when clustering infants by diet (breast-fed vs. formula-fed) compared to standard CP or functional SVD methods. By using Poisson and Beta divergence losses, the model successfully identified patterns in sparse count data that traditional methods—often requiring ad-hoc log-transformations—missed.

Conclusion and Future Work

This framework provides a robust solution for analyzing messy, real-world longitudinal data. While currently optimized for order-3 tensors, the authors plan to extend the framework to higher-order architectures like Tucker decomposition and Tensor-Train. Future work will also explore scenarios involving multiple functional (unaligned) modes and hybrid combinations of tabular and functional data.

Summary updated

BibTeX
@misc{tang2024tensor,
  author = {Runshi Tang and Tamara G. Kolda and Anru R. Zhang},
  title = {{Tensor Decomposition with Unaligned Observations}},
  year = {2024},
  eprint = {2410.14046},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.14046},
}
Figure