High-Order Tensor Representations for Multi-View Learning

This direction is merged into another direction: high-order tensor correlation. Its last version stays readable and citable here.

Version 1 of 130 papers32 conceptsfrozen 2026-10-04

In modern machine learning, we rarely describe an object with a single set of features. A news story consists of text, images, and video; a person is identified by their face, gait, and voice. These are "multi-view" data. The fundamental challenge is fusion: how do we integrate these diverse perspectives into a single, coherent representation?

Traditional approaches often treat this as a matrix problem. They might concatenate features into a long vector or perform pairwise comparisons between views. However, a growing body of research suggests that these methods are fundamentally limited because they ignore high-order correlations—the complex, multi-way dependencies that only emerge when all views are analyzed simultaneously.

To capture these relationships, researchers have turned to third-order tensors and the specialized algebra of the t-product.

The Matrix Bottleneck: Pairwise is Not Enough

Most classical multi-view methods, such as Canonical Correlation Analysis (CCA) or standard subspace clustering, operate on matrices. As Yuan Xie, Dacheng Tao, and colleagues point out in On Unifying Multi-View Self-Representations for Clustering by Tensor Multi-Rank Minimization, matrix-based methods typically analyze views in isolation or through simple linear combinations.

Imagine three people describing a crime scene. Person A saw the car, Person B saw the driver, and Person C heard the engine. A pairwise approach compares A to B, B to C, and A to C. While useful, this misses the "global" structure—the fact that the car, driver, and sound are all facets of a single event. By flattening data into 2D matrices, we lose the high-dimensional context required to find a consistent global representation (Weighted Low-Rank Tensor Representation for Multi-View Subspace Clustering).

The Tensor Solution: Stacking the Views

The core intuition behind this research direction is to represent multi-view data as a 3D tensor. If you have $N$ data points and $V$ views, you can construct a "self-representation" matrix for each view that describes how every point relates to every other point. By stacking these $V$ matrices on top of each other, you create a 3D block of data (a tensor of size $N \times N \times V$).

This structure allows researchers to apply tensor multi-rank minimization. In Low-Rank Tensor Constrained Multiview Subspace Clustering, Huazhu Fu and Xiaochun Cao demonstrate that by enforcing a low-rank constraint across the entire tensor, the model can automatically filter out view-specific noise and highlight the underlying consensus structure shared by all views.

Capturing Correlation via t-SVD

To make this tensor approach mathematically tractable, many of these papers rely on the tensor Singular Value Decomposition (t-SVD) and the t-product. Unlike traditional tensor decompositions (like CP or Tucker) which can be computationally unstable, t-SVD treats the 3D tensor as a matrix of "tubes," allowing for an algebraic framework that mimics standard matrix operations while preserving high-order spatial structures.

Junbin Gao and Ming Yin show in Low-rank Multi-view Clustering in Third-Order Tensor Space that this framework preserves the structural integrity of the data. By operating in the Fourier domain, t-SVD captures how information in one view "circulates" or correlates with others, leading to a more robust "consensus graph" than any single view could provide.

This mechanism has been extended to various domains:

  • Deep Learning: Hok Shing Wong and Raymond H. Chan introduced Deep Tensor CCA, which uses neural networks to learn nonlinear representations while maximizing high-order correlations through a covariance tensor.
  • Kernel Methods: Yanyun Qu and Wensheng Zhang combined the "kernel trick" with tensor minimization in Kt-SVD-MSC to handle non-linear relationships in complex real-world datasets.

Handling Incomplete Data

One of the most powerful applications of this tensor framework is in Incomplete Multi-View Clustering (IMVC). In real-world scenarios, some views are often missing (e.g., a patient has an MRI but no blood test).

Because the tensor framework models the global relationship across all views, it can "fill in" missing information more accurately than pairwise methods. Shuping Zhao and Jie Wen developed TIMVC_IGC, which treats the relationships as a 3D tensor to simultaneously recover missing data and discover a unified clustering consensus. Similarly, Tingting Wu and Jiazheng Yuan use a kernel tensor to capture high-order correlations that accurately fill missing values, outperforming methods that only consider local sample similarities.

Summary of Intuition

The shift from matrix-based to tensor-based multi-view learning represents a move from local, pairwise comparisons to global, holistic integration. By treating multiple views as a single 3D entity, these methods:

  1. Preserve Context: They avoid the information loss inherent in flattening or concatenating data.
  2. Exploit Redundancy: They use the low-rank nature of the shared "truth" across views to filter out noise.
  3. Unify Information: They provide a mathematically rigorous way (via t-SVD) to find a consensus that respects the unique contributions of every view simultaneously.

As demonstrated by Jianlong Wu and Zhouchen Lin in their work on UGLTL, this approach not only improves accuracy—sometimes achieving perfect scores on benchmarks—but also offers the scalability needed for the massive, multi-modal datasets of the modern era.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_high_order_tensor_representations_for_mu_fec7c434,
  title = {High-Order Tensor Representations for Multi-View Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/high-order-tensor-representations-for-multi-view-learning/rdr_fec7c434a0fb4113a568a45d19694077},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_high_order_tensor_representations_for_mu_fec7c434_v1,
  title = {High-Order Tensor Representations for Multi-View Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/high-order-tensor-representations-for-multi-view-learning/rdr_fec7c434a0fb4113a568a45d19694077/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}