Mode Coresets for Efficient, Interpretable Tensor Decompositions: An Application to Feature Selection in fMRI Analysis

Ben Gabrielson, Hanlu Yang, Trung Vu, Vince D. Calhoun, Tülay AdaliIEEE Access 2024 · DOI

TL;DR

This paper introduces "Tensor Coreset Decomposition" (TCD), a novel method for efficiently decomposing large, multi-dimensional datasets (tensors). TCD offers a balance between computational speed and accurate representation, and importantly, enables unsupervised feature selection, identifying meaningful components within the data, which is demonstrated on fMRI data.

What Problem Does This Paper Address?

Modern datasets are increasingly large and complex, often represented as tensors (multi-dimensional arrays). Analyzing these tensors requires tensor decomposition methods, which break down the large tensor into smaller, more manageable components. However, existing methods often face a trade-off: they are either computationally expensive, or they sacrifice accuracy for speed. Furthermore, many methods lack the ability to perform unsupervised feature selection, which is crucial for identifying the most informative parts of the data, especially in fields like neuroimaging where understanding specific functional networks is key. This paper addresses these challenges by proposing an efficient and interpretable tensor decomposition method that also excels at feature selection.

Methods

The core innovation is Tensor Coreset Decomposition (TCD). TCD is a type of Tucker decomposition that leverages "coresets" – weighted subsets of the tensor's elements. The method works by iteratively selecting coresets for each mode (dimension) of the tensor. These coresets are chosen to minimize a "discrepancy measure," which is directly related to the approximation error of the decomposition.

Two variants of TCD are proposed:

  • TCD-R (Randomized): Uses norm sampling to select subsets, prioritizing computational efficiency.
  • TCD-D (Deterministic): Employs a weighted kernel herding (WKH) algorithm for subset selection, prioritizing approximation quality and enabling feature selection.

TCD aims to approximate the High-Order Singular Value Decomposition (HOSVD) core tensor, which represents the principal components of the tensor. The method sequentially truncates the tensor by replacing it with the coreset, reducing computational complexity for subsequent steps. A key aspect is the use of learned weights for the coreset elements, which allows for better approximation than simple subset selection.

Datasets

The paper evaluates its methods on:

  • Simulated Data: Generated using both Canonical Polyadic Decomposition (CPD) and Tucker models to test performance under controlled conditions.
  • Real fMRI Data: Resting-state fMRI data from the bipolar-schizophrenia network on intermediate phenotypes (B-SNIP) dataset. This data consists of functional connectivity matrices (FNCs) from 352 subjects, forming a 53x53x352 tensor. Both the original FNC tensor and its elementwise-squared version were used.

Key Insights and Ideas

  • Coresets for Efficiency and Accuracy: Using weighted subsets (coresets) is a powerful strategy to achieve both computational efficiency and high approximation quality in tensor decomposition.
  • Deterministic Feature Selection: TCD-D's deterministic subset selection is a novel approach for unsupervised feature selection in tensors, identifying "well-representative" elements that are crucial for understanding the data's structure. This is particularly valuable in fMRI for identifying key functional networks.
  • Bridging the Gap: TCD effectively bridges the gap between purely random subset methods (fast but less accurate) and deterministic methods (accurate but potentially slow), offering a balanced solution.
  • Interpretability: The method's ability to select meaningful subsets (functional networks in the fMRI case) enhances the interpretability of the decomposition results.

Key Results

  • Computational Efficiency: TCD-R and TCD-D are shown to be among the most efficient methods, especially TCD-R, which offers competitive performance with significantly lower computational cost than many others. TCD-D also shows good efficiency, particularly for smaller tensor ranks.
  • Approximation Quality: TCD-D demonstrates competitive approximation error, often rivaling or exceeding other methods, and in some cases, closely matching ST-HOSVD despite using only a subset of the data.
  • Feature Selection in fMRI: On the fMRI dataset, TCD-D successfully identified a subset of functional networks that were highly representative of the overall connectivity patterns. These identified networks correspond to known brain regions crucial for neurological function and have been implicated in psychiatric disorders.
  • Robustness: The fMRI experiments showed that TCD-D consistently identified meaningful functional areas across both the original and elementwise-squared FNC tensors, highlighting the robustness of the method.

Limitations and Future Work

  • Computational Complexity for Large R: The paper notes that TCD-D's complexity increases with the estimated core size R, which might make other methods preferable when R is very large and CPU time is a strict priority.
  • Symmetry Handling: While the paper describes how to handle symmetric tensors, the primary focus is on asymmetric tensors.
  • Future Work: The authors suggest exploring more advanced subset selection schemes and further investigating the interpretability of the selected features, particularly in the context of complex biological data. The potential for TCD-D to perform unsupervised feature selection is highlighted as a key area for future research.

Summary updated

BibTeX
@article{gabrielson2024mode,
  author = {Ben Gabrielson and Hanlu Yang and Trung Vu and Vince D. Calhoun and Tülay Adali},
  title = {{Mode Coresets for Efficient, Interpretable Tensor Decompositions: An Application to Feature Selection in fMRI Analysis}},
  journal = {IEEE Access},
  year = {2024},
  doi = {10.1109/access.2024.3517338},
}
Figure