Abstract
High-dimensional data recovery often relies on tensor network decompositions with fixed topologies, which struggle with complex, unbalanced correlations. This paper proposes Multi-Tensor Network Representation (MTNR), a framework that models tensors as a superposition of multiple components with adaptively learned topologies. By dynamically building network structures from data, MTNR achieves superior performance in tensor completion tasks, particularly under extreme missing data conditions.
The Challenge of Fixed Topologies
Real-world high-dimensional data, such as color videos or multi-view images, are mathematically represented as tensors. These datasets often suffer from missing entries due to sensor failures or transmission errors. Tensor completion aims to fill these gaps by exploiting the data's inherent low-rank structure.
Traditional methods decompose a large tensor into a network of smaller, interconnected latent factors. Popular models like Tensor Train (TT) and Tensor Ring (TR) rely on rigid, pre-defined network topologies. However, real-world data is often "unbalanced"—some dimensions may be highly correlated while others are not. Fixed topologies struggle to capture these varying relationships and require difficult manual tuning of the "rank" (complexity) for each connection.
The MTNR Framework
To overcome these limitations, the authors introduce Multi-Tensor Network Representation (MTNR). Instead of a single fixed network, MTNR models a tensor as a superposition (sum) of multiple low-rank tensor network components.
The framework is powered by two primary innovations:
- Adaptive Topology Learning (ATL): Rather than assuming a structure, ATL builds it dynamically. It starts with a simple configuration and iteratively adds connections between latent factors if doing so significantly reduces reconstruction error. This allows the model to "discover" the optimal correlation structure within the data.
- Advanced Optimization Solvers: The authors developed MTNR-ALS and MTNR-ADMM to handle missing data. MTNR-ADMM is particularly efficient; it applies low-rank regularization directly to the latent factors. This mathematical approach keeps computational complexity linear relative to the tensor's order, preventing the exponential "curse of dimensionality."
Key Insights
- Data-Driven Flexibility: Letting the data dictate the network's shape allows MTNR to adapt to complex correlations that human-engineered models might miss.
- Permutation Invariance: Unlike TT or TR models, which can lose accuracy if the order of dimensions (modes) is shuffled, MTNR's dynamic topology makes it robust to how data is initially arranged.
- Latent Space Efficiency: The authors proved that complex multilinear operations can be performed directly in the latent factor space, allowing for the manipulation of multiple networks without instantiating the full, memory-intensive high-dimensional tensor.
Performance and Results
MTNR was evaluated against synthetic and real-world benchmarks, including color images (e.g., Lena, House) and the Extended Yale Face Database.
- Extreme Data Loss: In synthetic tests with a 99% missing data rate, MTNR successfully recovered the original structure where fixed-rank models failed.
- Image Inpainting: On benchmark images with 90% missing pixels, MTNR-ADMM consistently outperformed state-of-the-art methods. For the "House" image, it achieved a Peak Signal-to-Noise Ratio (PSNR) of 24.88, surpassing the popular TR-ALS method (24.11).
- Non-Random Missing Data: The model also excelled at recovering data with structured gaps (e.g., missing rows/columns), achieving a Structural Similarity (SSIM) of 0.9202 on the "House" image with 40% non-random missing data.
Conclusion
MTNR represents a significant shift from rigid tensor decomposition to flexible, data-driven representation. While the dynamic search for optimal topology introduces some computational overhead compared to fixed-topology methods, the gains in accuracy and robustness—especially in extreme or unbalanced data scenarios—make it a powerful tool for high-dimensional data recovery.
Summary updated