Abstract
A novel tensor decomposition framework, termed Tensor Star (TS) decomposition, is proposed which represents a new type of tensor network decomposition based on tensor contractions. This is achieved by connecting the core tensors in a ring shape, whereby the core tensors act as skip connections between the factor tensors and allow for direct correlation characterisation between any two arbitrary dimensions. Uniquely, this makes it possible to decompose an order-$N$ tensor into $N$ order-$3$ factor tensors {$\mathcal{G}_{k}$}$_{k=1}^{N}$ and $N$ order-$4$ core tensors {$\mathcal{C}_{k}$}$_{k=1}^{N}$, which are arranged in a star shape. Unlike the class of Tensor Train (TT) decompositions, these factor tensors are not directly connected to one another. The so obtained core tensors also enable consecutive factor tensors to have different latent ranks. In this way, the TS decomposition alleviates the "curse of dimensionality" and controls the "curse of ranks", exhibiting a storage complexity which scales linearly with the number of dimensions and as the fourth power of the ranks.
Introduction
In the analysis of multi-dimensional data, researchers face a persistent trade-off between expressive power and storage efficiency. Traditional methods like Tucker decomposition capture complex interactions between all dimensions but suffer from the "curse of dimensionality," where storage requirements grow exponentially with the number of dimensions. Conversely, efficient models like Tensor Train (TT) or Tensor Ring (TR) offer linear storage costs but are primarily "locally-sighted," capturing correlations mainly between adjacent dimensions. While newer frameworks like Fully-Connected Tensor Networks (FCTN) address these correlation issues, they often reintroduce high storage overheads.
The Tensor Star (TS) Decomposition
To bridge the gap between efficiency and expressivity, we introduce the Tensor Star (TS) decomposition. This framework represents a new type of tensor network that connects core tensors in a ring shape. In this configuration, core tensors act as "skip connections" between factor tensors, allowing for direct correlation characterization between any two arbitrary dimensions of a high-order tensor.
The TS architecture decomposes an order-$N$ tensor into:
- $N$ Factor Tensors ($\mathcal{G}_k$): Order-3 tensors that represent the individual dimensions (modes) of the original data.
- $N$ Core Tensors ($\mathcal{C}_k$): Order-4 tensors arranged in a central ring.
Unlike the TT decomposition where factors are linked in a linear chain, the TS factor tensors are connected to the core tensors, which are then linked to each other in a ring. This creates a "highway" for information flow, enabling any dimension to reach any other dimension through the central ring of cores. To make this practical, we developed two primary algorithms: TSD-ALS (Alternating Least Squares) for standard decomposition and TSD-PAM (Proximal Alternating Minimization) for robust tensor completion.
Key Insights and Advantages
The TS decomposition offers several theoretical and practical benefits:
- Direct Correlation: The ring topology of core tensors enables any two arbitrary dimensions to "talk" to each other directly, a significant upgrade over TT/TR models where information must pass through every intermediate dimension.
- Alleviating the "Curse of Ranks": In linear models, a high rank in one dimension forces neighboring dimensions to also use high ranks to pass information. TS avoids this by using the core ring to pass information, allowing consecutive factor tensors to maintain different latent ranks.
- Neural Network Compression: The "skip connection" nature of the core ring provides a path for gradients, potentially mitigating vanishing gradient issues during the training of compressed deep learning layers.
- Linear Scaling: The storage complexity is $O(NIR^2 + NR^4)$. While the $R^4$ term is higher than TT's $R^2$, the linear scaling with the number of dimensions ($N$) makes it viable for massive, high-order datasets.
Experimental Results
The framework was evaluated using a Multi-Spectral Image (MSI) dataset ($200 \times 200 \times 31$). In tensor completion tasks, the TSD-PAM algorithm successfully recovered high-quality images from only 10% of the original pixels (90% missing data). The model achieved a Mean Peak Signal-to-Noise Ratio (MPSNR) of 36.5466, demonstrating that the theoretical benefits of the "Star" structure translate into effective data reconstruction performance.
Conclusion
The Tensor Star decomposition provides a scalable and expressive alternative to existing tensor networks. By arranging tensors in a unique star-ring configuration, it achieves linear storage scaling while providing the connectivity needed for complex data representation. Future work will involve applying TS decomposition to actual Transformer or CNN weights to further verify its potential for deep learning model compression.
Summary updated