Abstract
Tensor rank learning for canonical polyadic decomposition (CPD) has long been deemed as an essential yet challenging problem. In particular, since the tensor rank controls the complexity of the CPD model, its inaccurate learning would cause overfitting to noise or underfitting to the signal sources, and even destroy the interpretability of model parameters. However, the optimal determination of a tensor rank is known to be a non-deterministic polynomial-time hard (NP-hard) task. Rather than exhaustively searching for the best tensor rank via trial-and-error experiments, Bayesian inference under the Gaussian-gamma prior was introduced in the context of probabilistic CPD modeling, and it was shown to be an effective strategy for automatic tensor rank determination. This triggered flourishing research on other structured tensor CPDs with automatic tensor rank learning. On the other side of the coin, these research works also reveal that the Gaussian-gamma model does not perform well for high-rank tensors and/or low signal-to-noise ratios (SNRs). To overcome these drawbacks, in this paper, we introduce a more advanced generalized hyperbolic (GH) prior to the probabilistic CPD model, which not only includes the Gaussian-gamma model as a special case, but also is more flexible to adapt to different levels of sparsity. Based on this novel probabilistic model, an algorithm is developed under the framework of variational inference, where each update is obtained in a closed-form. Extensive numerical results, using synthetic data and real-world datasets, demonstrate the significantly improved performance of the proposed method in learning both low as well as high tensor ranks even for low SNR cases.
Introduction
Tensor decomposition is a cornerstone of multi-dimensional data analysis, utilized in diverse fields such as drug discovery and social network mining. A fundamental challenge in Canonical Polyadic Decomposition (CPD) is determining the tensor rank ($R$), which represents the number of latent components. Selecting an incorrect rank leads to significant modeling issues: an over-estimated rank causes the model to overfit to noise, while an under-estimated rank misses critical signal components.
Determining the true rank is an NP-hard problem. While previous Bayesian methods employed "sparsity-promoting" priors, such as the Gaussian-gamma prior, to prune unnecessary components automatically, these methods are often too rigid. They struggle particularly when the true rank is high relative to the tensor's dimensions or in low Signal-to-Noise Ratio (SNR) environments. This paper introduces a robust Bayesian framework, PCPD-GH, which utilizes a flexible Generalized Hyperbolic (GH) prior to accurately identify tensor ranks in these challenging scenarios.
Methodology: The PCPD-GH Framework
The core innovation of the PCPD-GH model is the transition from a rigid Gaussian-gamma prior to the highly flexible Generalized Hyperbolic (GH) prior. The GH prior can be expressed as a "Gaussian scale mixture," allowing it to encompass many common distributions (e.g., Laplacian, Student’s t) as special cases. This flexibility enables the model to adapt its "peakiness" at zero and the "heaviness" of its tails to match the actual sparsity level of the data.
Because the exact posterior distribution is mathematically intractable, the authors derive a Variational Inference (VI) algorithm. This provides a way to approximate the complex distribution using simpler, closed-form update equations. The algorithm typically starts with an over-estimated rank; during the iterative learning process, the GH prior drives the power of redundant columns in the factor matrices to near-zero. These "empty" columns are then pruned on-the-fly, leaving behind the true estimated rank. Furthermore, the model optimizes the most critical hyper-parameter ($a_l^0$) within the VI framework, enhancing its adaptivity across different datasets.
Experimental Evaluation and Results
The researchers evaluated PCPD-GH using synthetic simulations and diverse real-world datasets, including fluorescence data, the ENRON email dataset, and hyperspectral images (HSI).
Key results from synthetic tests demonstrate that in high-rank scenarios (e.g., true rank of 24), PCPD-GH achieved significantly higher accuracy than standard methods. For instance, at 10 dB SNR, the proposed method accurately identified the rank where the standard Gaussian-gamma method failed. Even at 0 dB SNR, the method maintained high accuracy, whereas competing methods either over-estimated or under-estimated the rank.
In real-world applications, PCPD-GH correctly identified the three underlying chemical components in a fluorescence dataset even at -10 dB SNR. In hyperspectral image denoising, it consistently outperformed benchmark Bayesian methods in terms of output SNR. A major practical advantage is the model's robustness to initialization; users can start with a very large initial rank estimate, and the model will reliably converge to the correct value without the performance degradation seen in older methods.
Conclusion
By replacing rigid priors with the flexible Generalized Hyperbolic distribution, the PCPD-GH framework overcomes long-standing limitations in automatic tensor rank learning. While the use of complex mathematical functions like modified Bessel functions slightly increases computational complexity per iteration, the gains in accuracy and robustness—especially for high-rank and noisy data—are substantial. Future work may involve integrating this GH prior into non-negative tensor factorization or deep-learning-based decomposition models to further enhance their structural learning capabilities.
Summary updated