Joint Tensor Factorization and Outlying Slab Suppression with Applications

Xiao Fu, Kejun Huang, Wing‐Kin Ma, Nicholas D. Sidiropoulos, Rasmus BroIEEE TSP 2015

Introduction

In multi-dimensional data analysis, tensors serve as the primary mathematical structure for capturing complex, multi-way relationships. However, real-world datasets—ranging from chemical spectroscopy to social network logs—are frequently contaminated by outliers. Standard tensor factorization techniques, such as Canonical Polyadic Decomposition (CPD), are highly sensitive to these "bad apples," often leading to skewed results. This paper introduces a robust framework for tensor factorization that specifically targets "slab-level" corruption.

The Challenge of Slab-Level Corruption

Most existing robust methods address outliers at the element-wise level. However, in many practical scenarios, corruption occurs across entire 2D slices of a 3D tensor, known as "slabs." For instance, a malfunctioning sensor might corrupt an entire time frame in a multi-channel recording, or a door slam might ruin a specific time segment in speech processing.

A 3D tensor $\underline{\mathbf{X}}$ with dimensions $I, J, K$, representing the multi-way data structure.

Standard least-squares approaches fail in these cases because they attempt to minimize the squared error of the corrupted slabs, pulling the entire model away from the true underlying factors. Prior attempts to resolve this were often computationally prohibitive or lacked convergence guarantees.

The IRALS Framework

The authors propose Iteratively Reweighted Alternating Least Squares (IRALS), an algorithm designed to identify and downweight these corrupted slabs automatically. The core innovation lies in framing slab-level robustness as a group-sparsity problem. By employing an $\ell_p$ quasi-norm ($0 < p \le 1$) on the slab-wise errors, the objective function promotes sparsity in the error terms. This effectively encourages the model to treat most slabs as clean (zero error) while isolating and suppressing outliers.

Visualizing different slab orientations ($\mathbf{X}_i^{(3)}$ and $\mathbf{X}_j^{(1)}$) that the algorithm processes to identify mode-specific corruption.

The IRALS algorithm integrates the Alternating Direction Method of Multipliers (ADMM) framework, allowing it to incorporate physical constraints such as non-negativity and smoothness. Crucially, IRALS maintains a computational complexity similar to standard non-robust methods, making it significantly faster (up to 50 times in simulations) than previous $\ell_1$-norm based approaches.

Experimental Validation and Results

The effectiveness of IRALS was demonstrated across several distinct domains:

  • Blind Speech Separation: In noisy environments with more speakers than sensors (under-determined scenarios), IRALS achieved a 15dB improvement in Mean Squared Error (MSE) over existing state-of-the-art methods like SOBIUM.
  • Fluorescence Spectroscopy: Analyzing the classic "Dorrit" dataset, the algorithm successfully identified and suppressed a known outlier slab (Slab 5), resulting in chemical signatures that closely matched pure laboratory samples.
  • Social Network Mining: When applied to the ENRON email corpus, IRALS automatically downweighted the months associated with the company's bankruptcy and federal investigation. This allowed the model to extract more stable and logical social groupings by ignoring the irregular communication patterns of the crisis period.

Conclusion

The IRALS algorithm provides a computationally efficient and robust solution for tensor factorization in the presence of slab-level outliers. By leveraging group-sparsity and iterative reweighting, it offers a powerful tool for researchers dealing with messy, high-dimensional data. Future work includes developing multi-mode robust versions and further refining the convergence theory for non-convex objectives.

Summary updated

Abstract

We consider factoring low-rank tensors in the presence of outlying slabs. This problem is important in practice, because data collected in many real-world applications, such as speech, fluorescence, and some social network data, fit this paradigm. Prior work tackles this problem by iteratively selecting a fixed number of slabs and fitting, a procedure which may not converge. We formulate this problem from a group-sparsity promoting point of view, and propose an alternating optimization framework to handle the corresponding $\ell_p$ ($0
BibTeX
@article{fu2015joint,
  author = {Xiao Fu and Kejun Huang and Wing‐Kin Ma and Nicholas D. Sidiropoulos and Rasmus Bro},
  title = {{Joint Tensor Factorization and Outlying Slab Suppression with Applications}},
  journal = {IEEE Trans. Signal Process.},
  year = {2015},
}
Figure