Abstract
This paper presents novel incremental update techniques for parallel Generalized Canonical Polyadic (GCP) and Tucker tensor decompositions, optimized for high-performance computing (HPC) architectures. Implemented within the GenTen 4 library, these methods facilitate the real-time analysis of massive, streaming multi-dimensional data by processing updates in batches. This approach significantly mitigates memory overhead while achieving accuracy levels comparable to static models that require full dataset access.
Introduction
Tensors are essential for representing multi-dimensional data in fields such as signal processing and fluid dynamics. Traditional decomposition methods assume the availability of the entire dataset, which is often impractical for continuous data streams or datasets exceeding memory capacity. Recomputing decompositions from scratch for every new data increment is computationally unfeasible. We address this by developing "online" algorithms capable of ingesting new data, updating models incrementally, and discarding historical raw data without sacrificing pattern recognition capabilities.
Methodology
Streaming Generalized CP (GCP) for Sparse Tensors
Unlike standard CP models that assume Gaussian noise, our streaming GCP approach supports diverse loss functions (Normal, Poisson, Bernoulli), making it suitable for sparse count or binary event data.
The method utilizes a two-step optimization via Stochastic Gradient Descent (SGD). It first solves for the current temporal factor and then updates global non-temporal factors. By employing the ADAM optimizer with a selective reset strategy—resetting moments for rapidly changing temporal factors while preserving them for stable non-temporal factors—the model maintains responsiveness to new trends without excessive oscillation.
Streaming Tucker Decompositions for Dense Tensors
For dense datasets typical in scientific simulations, we introduce a streaming Tucker decomposition based on Sequentially Truncated Higher Order SVD (ST-HOSVD). This technique updates factor matrices and the core tensor incrementally. It allows the model's rank to evolve dynamically, ensuring reconstruction error remains within specified tolerances. The core is updated through row/column augmentation, ensuring the algorithm never needs to re-process old data.
Experimental Results
The methods were evaluated using real-world datasets including NYC Taxicab trips, ArXiv abstracts, and Chicago Crime reports, as well as simulated HCCI combustion data.
Performance and Memory Efficiency
In sparse data evaluations, the Online GCP method tracked the performance of the "Static GCP" gold standard with high fidelity. For the Chicago Crime dataset, the incremental updates maintained model quality without significant drift.
In dense data applications, the streaming Tucker algorithm achieved a relative error of $10^{-3}$ on the HCCI combustion dataset using 16-22 GB of memory, compared to 38 GB for the batch equivalent. These implementations in GenTen and TuckerMPI support both shared-memory and distributed-memory architectures, demonstrating readiness for HPC environments.
Conclusion
The proposed incremental update techniques provide a robust framework for online tensor analysis. By differentiating the evolution of temporal and non-temporal factors and enabling dynamic rank adjustments, these methods achieve a balance between memory efficiency and analytical accuracy. Future research will explore further hardware optimizations and automated parameter tuning.
Summary updated