A Statistically Identifiable Model for Tensor-Valued Gaussian Random Variables

Bruno Scalzo Dees, Danilo P. MandicarXiv 2019 · arXiv

Abstract

This paper introduces a statistically identifiable Gaussian distribution framework specifically designed for tensor-valued data (multi-way arrays). By enforcing unique constraints on model parameters, the authors move away from slow, iterative estimation algorithms toward a rigorous, closed-form analytical solution. The resulting model is computationally efficient, statistically consistent, and provides high physical interpretability, as demonstrated through climate data modeling.

Introduction

In fields such as climate science and medical imaging, data naturally exists as tensors—multi-dimensional grids of measurements. Traditional multivariate statistics often "flatten" these structures into large vectors, leading to the "curse of dimensionality" where the number of parameters explodes, making estimation difficult without massive sample sizes. While existing tensor models use "Kronecker separability" to reduce parameters, they often suffer from non-identifiability (multiple parameter sets describing the same distribution) and rely on iterative "flip-flop" algorithms that may converge to local optima.

Methodology and Identifiability

The authors propose a representation-invariant formulation of the tensor-valued Gaussian distribution. The core innovation lies in the introduction of identifiability constraints:

  • Mean Vectors: Mode-specific mean vectors are constrained to unit norm (||μ⁽ⁿ⁾|| = 1).
  • Covariance Matrices: Mode-specific covariance matrices are constrained to unit trace (tr(Θ⁽ⁿ⁾) = 1).

These constraints ensure a one-to-one mapping between the probability distribution and its parameters. Consequently, the authors derive a closed-form Maximum Likelihood Estimation (MLE) solution, allowing optimal parameters to be calculated directly from sample statistics without iterative searching.

Illustration of tensor matricization (unfolding) across three modes, a key operation in tensor-valued statistical modeling.

Key Insights: Parameter Reduction and Interpretability

The proposed model achieves massive parameter reduction by leveraging the tensor structure. For instance, in modeling global atmospheric temperature data (a 721 × 1440 × 20 grid), the number of parameters for the mean was reduced from approximately 20.7 million to just 2,182.

Because the model is identifiable, the estimated factors are physically interpretable. In climate applications, the mode-1 factor captures temperature variation by latitude, while the mode-3 factor reflects the expected temperature drop with increasing altitude.

Representation of a rank-1 tensor as the outer product of three vectors, illustrating the separable mean structure proposed in the model.

Results and Validation

The model was validated using both synthetic Monte Carlo simulations and real-world data from the ECMWF ERA5 dataset.

  • Consistency: Simulations confirmed that the estimator is statistically consistent, with estimation variance decreasing linearly as the sample size increases.
  • Accuracy: The rank-1 tensor mean reconstruction explained 99.99% of the variance in atmospheric temperature sample means, validating the "separable" assumption as a physically accurate representation.
  • Efficiency: The model remains effective even with very small sample sizes, provided the tensor dimensions are sufficiently large.

Limitations and Future Work

The current model assumes a rank-1 structure for the mean, which may not suit highly complex or "jagged" datasets. Future research could explore higher-rank approximations and extend the identifiable framework to non-Gaussian distributions, such as Heavy-Tailed or Student-t models, to better accommodate outliers and non-stationary processes.

Conclusion

By establishing a statistically identifiable framework with a closed-form MLE, this work provides a robust and efficient tool for tensor-valued data analysis, offering significant improvements in both computational speed and physical interpretability over traditional iterative methods.

Summary updated

BibTeX
@misc{dees2019statistically,
  author = {Bruno Scalzo Dees and Danilo P. Mandic},
  title = {{A Statistically Identifiable Model for Tensor-Valued Gaussian Random Variables}},
  year = {2019},
  eprint = {1911.02915},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1911.02915},
}
Figure