Group Sparse-based Tensor CP Decomposition: Model, Algorithms, and Applications in Chemometrics

Zihao Wang, Minru Bai, Liang Chen, Xueying Zhao

Abstract

The CANDECOMP/PARAFAC (CP) decomposition is a cornerstone of multi-way data analysis, yet its practical application is often hindered by the requirement to know the exact CP rank in advance. Because determining the rank of a tensor is an NP-hard problem, traditional methods frequently suffer from factor degeneracy or poor fit due to incorrect rank estimation. This paper introduces a novel CP decomposition model that integrates rank estimation and decomposition into a single optimization problem using group sparse regularization. By employing a double-loop block-coordinate proximal gradient descent algorithm with a rank reduction strategy, the proposed method effectively "prunes" unnecessary components during the optimization process. Experimental results on real-world chemometric datasets demonstrate that this approach is robust to rank overestimation and significantly more efficient than traditional decomposition algorithms.

Introduction: The Challenge of CP Rank Estimation

In fields ranging from signal processing to machine learning, tensors provide a natural way to represent multi-dimensional data. The CP decomposition breaks these complex tensors into a sum of rank-one components, allowing researchers to identify underlying structures—such as individual chemical signatures in a mixture.

However, a fundamental bottleneck exists: standard CP decomposition algorithms require the user to specify the CP rank (the number of components) beforehand. If the rank is underestimated, the model fails to capture the full complexity of the data. If it is overestimated, the model creates "ghost" components, leading to a phenomenon known as factor degeneracy where components become highly correlated and physically meaningless. Since calculating the exact rank is computationally intractable, researchers typically rely on trial-and-error. This paper addresses this "chicken-and-egg" problem by proposing a model that discovers the rank dynamically during the decomposition process.

Proposed Methodology: CPD_GSU and Rank Reduction

The authors propose a model termed CPD_GSU (CP Decomposition with Group Sparse regularization and Unit length constraints). The methodology is built upon three primary pillars:

1. The Group Sparsity Framework

The researchers prove a critical theoretical result: the CP rank of a tensor can be exactly estimated by minimizing the group sparsity of one factor matrix, provided the columns of the remaining factor matrices are constrained to unit length. By applying an $\ell_{2,0}$ norm penalty to the columns of the final factor matrix, the model encourages unnecessary components to shrink to zero. This transforms rank estimation from a discrete combinatorial problem into a continuous optimization problem.

2. The eDLBCPGD Algorithm

To solve the resulting non-convex optimization problem, the authors developed a double-loop block-coordinate proximal gradient descent algorithm.

  • Outer Loop: Monitors the overall progress and ensures the objective function is decreasing.
  • Inner Loop: Solves sub-problems for each factor matrix using extrapolation techniques to accelerate convergence.

3. Rank Reduction Strategy (RR)

A key innovation in this work is the Rank Reduction strategy. During the iterative process, if the algorithm determines that a specific column (component) has been reduced to zero and remains stable, that column is physically removed from the matrices. This prevents the algorithm from wasting computational resources on "dead" components, leading to faster convergence as the effective rank of the model decreases toward the true rank.

Experimental Validation in Chemometrics

The model was tested using real-world chemical datasets where identifying the exact number of chemical components is vital for accurate analysis:

  • Two-component mixture: A $32 \times 39 \times 15$ tensor involving chemical drift.
  • Macrocephalae rhizoma data: A $52 \times 214 \times 16$ tensor containing three distinct chemical components.

Key Findings

  • Robustness to Overestimation: In scenarios where the initial rank was set significantly higher than the truth (e.g., starting with $R=7$ for a 3-component system), traditional methods like CP_ALS and AIBCD failed to separate the components correctly. In contrast, the CPD_GSU model successfully pruned the four excess components and recovered the true chemical signatures.
  • Computational Efficiency: The rank reduction strategy proved highly effective. In the Macrocephalae rhizoma experiments, the version of the algorithm with rank reduction (eDLBCPGD_RR) was approximately twice as fast as the version without it, as it progressively simplified the problem size.
  • Accuracy: The proposed method achieved lower Root Mean Squared Error of Prediction (RMSEP) compared to industry-standard benchmarks, demonstrating that the group-sparse approach does not sacrifice precision for its automated rank-finding capabilities.

Conclusion and Future Work

This research provides a robust theoretical and practical framework for automated tensor decomposition. By reframing CP rank estimation as a group sparsity problem, the authors have eliminated the need for manual rank tuning, which is often the most time-consuming part of tensor analysis.

Future research directions include adapting the model for "big data" contexts involving massive, sparse tensors common in social network analysis, and integrating non-negativity constraints to better suit specific physical sciences where negative concentrations or intensities are impossible. Nevertheless, the current CPD_GSU model represents a significant step forward in making tensor decomposition a more accessible and automated tool for scientific discovery.

Summary updated

Abstract

The CANDECOMP/PARAFAC (or Canonical polyadic, CP) decomposition of tensors has numerous applications in various fields, such as chemometrics, signal processing, machine learning, etc. Tensor CP decomposition assumes the knowledge of the exact CP rank, i.e., the total number of rank-one components of a tensor. However, accurately estimating the CP rank is very challenging. In this work, to address this issue, we prove that the CP rank can be exactly estimated by minimizing the group sparsity of any one of the factor matrices under the unit length constraints on the columns of the other factor matrices. Based on this result, we propose a CP decomposition model with group sparse regularization, which integrates the rank estimation and the tensor decomposition as an optimization problem, whose set of optimal solutions is proved to be nonempty. To solve the proposed model, we propose a double-loop block-coordinate proximal gradient descent algorithm with extrapolation and prove that each accumulation point of the sequence generated by the algorithm is a stationary point of the proposed model. Furthermore, we incorporate a rank reduction strategy into the algorithm to reduce the computational complexity. Finally, we apply the proposed model and algorithms to the component separation problem in chemometrics using real data. Numerical experiments demonstrate the robustness and effectiveness of the proposed methods.
BibTeX
@misc{wang2025group,
  author = {Zihao Wang and Minru Bai and Liang Chen and Xueying Zhao},
  title = {{Group Sparse-based Tensor CP Decomposition: Model, Algorithms, and Applications in Chemometrics}},
  year = {2025},
  eprint = {2501.03776},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2501.03776},
}
Figure