Incremental update techniques for online analysis of streaming tensor data

Eric Phipps, Saibal De, Hemanth Kolla, Nicholas Johnson, Tamara G. Kolda, Zitong Li2023 · DOI

Abstract

This paper presents novel incremental update techniques for parallel Generalized Canonical Polyadic (GCP) and Tucker tensor decompositions, optimized for high-performance computing (HPC) architectures. Implemented within the GenTen 4 library, these methods facilitate the real-time analysis of massive, streaming multi-dimensional data by processing updates in batches. This approach significantly mitigates memory overhead while achieving accuracy levels comparable to static models that require full dataset access.

Introduction

Tensors are essential for representing multi-dimensional data in fields such as signal processing and fluid dynamics. Traditional decomposition methods assume the availability of the entire dataset, which is often impractical for continuous data streams or datasets exceeding memory capacity. Recomputing decompositions from scratch for every new data increment is computationally unfeasible. We address this by developing "online" algorithms capable of ingesting new data, updating models incrementally, and discarding historical raw data without sacrificing pattern recognition capabilities.

Methodology

Streaming Generalized CP (GCP) for Sparse Tensors

Unlike standard CP models that assume Gaussian noise, our streaming GCP approach supports diverse loss functions (Normal, Poisson, Bernoulli), making it suitable for sparse count or binary event data.

Streaming tensor data represented as a sequence of slices.

The method utilizes a two-step optimization via Stochastic Gradient Descent (SGD). It first solves for the current temporal factor and then updates global non-temporal factors. By employing the ADAM optimizer with a selective reset strategy—resetting moments for rapidly changing temporal factors while preserving them for stable non-temporal factors—the model maintains responsiveness to new trends without excessive oscillation.

Streaming Tucker Decompositions for Dense Tensors

For dense datasets typical in scientific simulations, we introduce a streaming Tucker decomposition based on Sequentially Truncated Higher Order SVD (ST-HOSVD). This technique updates factor matrices and the core tensor incrementally. It allows the model's rank to evolve dynamically, ensuring reconstruction error remains within specified tolerances. The core is updated through row/column augmentation, ensuring the algorithm never needs to re-process old data.

Experimental Results

The methods were evaluated using real-world datasets including NYC Taxicab trips, ArXiv abstracts, and Chicago Crime reports, as well as simulated HCCI combustion data.

Performance and Memory Efficiency

In sparse data evaluations, the Online GCP method tracked the performance of the "Static GCP" gold standard with high fidelity. For the Chicago Crime dataset, the incremental updates maintained model quality without significant drift.

Loss comparison for Online vs. Static GCP on Chicago Crime data.

In dense data applications, the streaming Tucker algorithm achieved a relative error of $10^{-3}$ on the HCCI combustion dataset using 16-22 GB of memory, compared to 38 GB for the batch equivalent. These implementations in GenTen and TuckerMPI support both shared-memory and distributed-memory architectures, demonstrating readiness for HPC environments.

Conclusion

The proposed incremental update techniques provide a robust framework for online tensor analysis. By differentiating the evolution of temporal and non-temporal factors and enabling dynamic rank adjustments, these methods achieve a balance between memory efficiency and analytical accuracy. Future research will explore further hardware optimizations and automated parameter tuning.

Summary updated

Abstract

Approach implemented in GenTen 4 , for parallel GCP decompositions targeting HPC architectures Shared memory parallelism (CPU
BibTeX
@misc{phipps2023incremental,
  author = {Eric Phipps and Saibal De and Hemanth Kolla and Nicholas Johnson and Tamara G. Kolda and Zitong Li},
  title = {{Incremental update techniques for online analysis of streaming tensor data}},
  year = {2023},
  doi = {10.2172/2431786},
}
Figure