Low-Rank Tensor Thresholding Ridge Regression

Kailing Guo, Tong Zhang, Xiangmin Xu, Xiaofen XingIEEE Access 2019 · DOI

Low-Rank Tensor Thresholding Ridge Regression (LR-TTRR) for Subspace Clustering

Subspace clustering is a fundamental task in computer vision, yet traditional methods often suffer from spatial information loss due to vectorization and inadequate noise handling. This paper presents Low-Rank Tensor Thresholding Ridge Regression (LR-TTRR), a novel approach that leverages tensor algebra to preserve multi-dimensional data structures. By implementing dual-space denoising and fixed-rank factorization, LR-TTRR achieves state-of-the-art accuracy and significant computational speedups compared to both traditional and deep learning-based methods.

The Problem: Spatial Destruction and Residual Noise

Subspace clustering aims to group data points based on the underlying low-dimensional subspaces they inhabit. A common technique is "self-representation," where each data point is modeled as a linear combination of other points in the dataset. However, conventional algorithms face two critical limitations:

  1. Spatial Destruction: Most algorithms vectorize 2D images into long 1D strings. This "flattening" destroys the spatial relationships between pixels (e.g., a pixel's neighbor is no longer its neighbor in the vector).
  2. Incomplete Noise Removal: While many methods address noise in the input data, they often neglect noise in the projection space (representation coefficients), leading to "dirty" affinity matrices that confuse the final clustering step.

The LR-TTRR Framework

To address these challenges, LR-TTRR shifts the mathematical foundation from matrix to tensor algebra. This transition allows the model to treat data as multi-dimensional tensors, preserving the inherent 2D topology of images. The core of LR-TTRR lies in its use of the tensor mode-d product. Unlike standard matrix multiplication, this operation allows the model to interact with different dimensions of the image (e.g., rows and columns) independently.

Comparison of TRR, VR3, and the proposed LR-TTRR architectures.

As shown in the figure above, LR-TTRR (c) utilizes sequential mode-unfolding and folding operations to maintain structural integrity throughout the representation process. To ensure efficiency, the authors employ fixed-rank factorization, decomposing representation matrices into smaller components. This avoids the high computational cost of nuclear norm minimization common in other methods.

Furthermore, LR-TTRR introduces a dual-space denoising strategy:

  • Input Space Denoising: A combination of Frobenius and $\ell_1$ norms is used to mitigate both Gaussian noise and sparse outliers.
  • Projection Space Denoising: A modified hard-thresholding strategy is applied to the affinity matrix. By retaining only the $k$ largest coefficients and zeroing the rest, the model prunes weak, noisy connections between unrelated data points, resulting in a cleaner representation for the final clustering step.

Experimental Results

The model was rigorously tested on benchmark datasets including Extended Yale B, ORL, COIL20/100, and Alphadigits.

  • Superior Accuracy: LR-TTRR consistently outperformed existing methods. On the ORL face dataset, it achieved a clustering error of 9.75%, surpassing traditional LRR (14.23%) and deep learning-based DSC-Net (11%). On the COIL20 object dataset, the error rate dropped to a remarkable 1.46%, significantly better than DSC-Net's 5.14%.
  • Massive Speedups: One of the most striking results is the computational efficiency. On the Extended Yale B dataset, previous spatial methods like VR3 required over three days to process. In contrast, LR-TTRR completed the same task in approximately 700 seconds (under 12 minutes) without requiring a GPU.

Conclusion

LR-TTRR demonstrates that preserving the 2D topology of image data through tensor algebra provides a richer feature set for clustering. By combining this structural preservation with aggressive thresholding in the projection space, the model achieves high accuracy without the computational overhead of deep learning. While hyperparameter tuning remains a challenge, LR-TTRR offers a powerful and efficient alternative for high-dimensional subspace clustering tasks.

Summary updated

BibTeX
@article{guo2019low,
  author = {Kailing Guo and Tong Zhang and Xiangmin Xu and Xiaofen Xing},
  title = {{Low-Rank Tensor Thresholding Ridge Regression}},
  journal = {IEEE Access},
  year = {2019},
  doi = {10.1109/access.2019.2944426},
}
Figure