Tensorized Multi-view Representation Learning

This direction is merged into another direction: high-order tensor correlation. Its last version stays readable and citable here.

Version 1 of 1104 papers149 conceptsfrozen 2026-10-04

In modern data science, we rarely observe a phenomenon through a single lens. A patient’s health is reflected in MRI scans, genetic markers, and clinical notes; a social media post consists of text, images, and metadata. Multi-view learning aims to integrate these diverse "views" to find a robust, underlying consensus.

Traditionally, researchers handled this by flattening data into matrices—either by concatenating features or averaging similarity graphs. However, as Jing Li and Quanxue Gao argue in their work on Orthogonal Non-negative Tensor Factorization, this 2D approach destroys the complex spatial structures and inter-view relationships inherent in the data. The emerging consensus across 81 papers is that multi-view data is best treated as a high-order tensor, preserving the multi-dimensional "cross-talk" that matrix-based methods ignore.

The Structural Advantage of Tensors

The core intuition behind this research direction is that views are not just independent lists of features; they are different slices of the same object. By stacking these slices into a 3rd-order tensor (Objects × Features/Anchors × Views), models can apply constraints across all dimensions simultaneously.

As Huazhu Fu and colleagues demonstrated with LT-MSC, treating subspace representations as a unified tensor allows for global low-rank constraints. While a matrix-based approach might look for patterns within a single view, a tensor-based approach looks for patterns that persist across the entire "stack." This captures high-order correlations—synergies where the information in one view (e.g., an image) helps resolve the noise or ambiguity in another (e.g., a text tag).

Capturing Synergy via Tensor Decomposition

How do we mathematically extract this consensus? The field has moved toward sophisticated tensor norms and decompositions that go beyond simple pairwise comparisons.

  1. Tensor-SVD (t-SVD): Many researchers, including Quanxue Gao in Weighted Tensor-Nuclear Norm Minimization, utilize the tensor singular value decomposition. By performing SVD in the Fourier domain, these models can efficiently capture relationships across all views.
  2. Weighted Schatten p-norm: To better handle noise, Wei Xia introduced the Weighted Tensor Schatten p-norm. This allows the model to prioritize significant singular values (the "signal") while suppressing smaller ones (the "noise"), leading to much sharper cluster separation.
  3. Tucker and CP Decomposition: For multimodal fusion, such as combining audio, video, and text, Peyman Momeni showed that Tucker decomposition allows for modality-specific compression rates, capturing non-linear dependencies that a simple diagonal matrix cannot.

Solving the "Missing View" Problem

One of the most powerful applications of this framework is handling incomplete data. In real-world datasets, it is common for a sample to be missing one or more views (e.g., a patient who has an MRI but no PET scan).

Jie Wen and Shuping Zhao have developed unified frameworks like IMVTSC-MVI and TIMVC_IGC that treat missing views as a tensor completion problem. Because the tensor structure enforces a global consensus, the model can "fill in the blanks" by leveraging the high-order correlations from the views that are present. The tensor acts as a 3D puzzle where the shape of the existing pieces constrains what the missing pieces must look like.

Scaling to Big Data

A common critique of tensor methods is their computational cost. However, recent innovations have made these techniques viable for large-scale datasets.

  • Anchor-based Graphs: Instead of building a full $N \times N$ similarity matrix for $N$ samples, researchers like Jing Li use a small set of "anchors" to represent the data structure, reducing complexity from $O(N^2)$ to $O(N)$.
  • Tensor Projection: Xingfeng Li proposed TPCH, which treats projection matrices themselves as a tensor. This allows for "cooperative hashing," where the transformation of one view is mathematically informed by the others, resulting in speeds up to 10x faster than traditional methods.

Why It Works: The Intuition of Multi-way Consistency

The success of these methods across domains—from Alzheimer's diagnosis by Changqing Zhang to detecting mental disorders via social media logs by Hong-Han Shuai—stems from a single principle: multi-way consistency.

While two views might look different on the surface, they share a latent structure. Matrix methods try to find this structure by looking at views in pairs. Tensor methods look at all views at once, identifying the "mathematical consensus" that emerges only when the data is viewed as a unified, high-dimensional whole. This holistic perspective makes the resulting representations significantly more robust to noise, outliers, and missing information.

Papers in this version

Showing 60 of 104 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_tensorized_multi_view_representation_lea_f5df0c72,
  title = {Tensorized Multi-view Representation Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/tensorized-multi-view-representation-learning/rdr_f5df0c72539345829ffd1306b7179339},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_tensorized_multi_view_representation_lea_f5df0c72_v1,
  title = {Tensorized Multi-view Representation Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/tensorized-multi-view-representation-learning/rdr_f5df0c72539345829ffd1306b7179339/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}