Orthogonal Equivalence, Singular Values and T-Rank of Third Order Tensors.

Liqun Qi, Ling Chen, Jinejie Liu, Chen Ouyang

In the landscape of linear algebra, the Singular Value Decomposition (SVD) is often described as the "Swiss Army Knife" of data analysis. It provides a robust framework for dimensionality reduction, noise filtering, and feature extraction in 2D datasets. However, as modern data increasingly takes the form of multi-dimensional arrays—such as video sequences, seismic data, or multi-view images—the need for a mathematically rigorous extension of the SVD to tensors becomes paramount.

In the paper "Orthogonal Equivalence, Singular Values and T-Rank of Third Order Tensors," the authors address critical theoretical gaps in tensor algebra. Building upon the T-product framework, they establish an Orthogonal Equivalence Theorem that ensures the stability and consistency of tensor singular values, providing a formal foundation for low-rank tensor approximation.

The Theoretical Framework: Kilmer-Martin Mapping

The foundation of this work lies in the T-product, a specialized multiplication operation for third-order tensors. To analyze these structures, the authors utilize the Kilmer-Martin mapping, a three-stage process designed to extract singular values from a 3D structure:

  1. Transformation: The tensor is moved into the Fourier domain using a Discrete Fourier Transform (DFT) along its tubes (the fibers extending through the third dimension).
  2. Decomposition: A standard matrix SVD is performed on each frontal slice of the transformed tensor. Crucially, the resulting singular values are sorted in a non-increasing order to ensure a canonical representation.
  3. Inversion: An Inverse DFT (IDFT) is applied to return the data to the original tensor space.

The Orthogonal Equivalence Theorem

A central contribution of this research is the formalization of Orthogonal Equivalence. In matrix theory, two matrices are orthogonally equivalent if one can be transformed into the other via multiplication by orthogonal matrices—a process that preserves singular values.

The authors prove that this property holds for third-order tensors. Theorem 3.2 (the "Main Event" of the paper) demonstrates that if two tensors are orthogonally equivalent (connected via T-product with orthogonal tensors), their Kilmer-Martin mappings are identical. This "Invariance Principle" confirms that the singular values extracted via T-SVD are intrinsic properties of the data itself, rather than artifacts of the tensor's orientation or coordinate system.

Redefining Rank: T-Rank vs. Tubal Rank

The paper introduces a refined definition of "T-rank." While "tubal rank" has been a standard measure in previous literature, the authors define T-rank as the total number of non-zero singular values across all slices in the Fourier domain. This provides a more granular understanding of the tensor's internal structure.

Interestingly, the authors demonstrate that T-rank does not follow the sub-additivity rule found in matrix algebra. In standard linear algebra, the rank of a sum is less than or equal to the sum of the ranks ($rank(A+B) \le rank(A) + rank(B)$). The authors provide counter-examples showing that for tensors, the T-rank of a sum can actually exceed the sum of the individual T-ranks, highlighting the unique complexity of higher-order data structures.

Key Insights and Approximations

Beyond theoretical definitions, the paper offers practical insights into how we can approximate complex data:

  • Best T-Rank One Approximation: The authors prove that the optimal way to approximate a tensor using a single "component" is to utilize the largest singular value and its associated orthogonal tensors derived from the T-SVD. This is a vital proof for data compression algorithms.
  • The Largest Entry Property: They extend a classic matrix property to tensors, proving that the largest singular value ($\sigma_1$) is always greater than or equal to the absolute value of any individual entry in the tensor ($\sigma_1(\mathcal{A}) \ge \max |a_{ijk}|$).

Limitations and Future Research

While this work provides a rigorous mathematical bridge between 2D and 3D data analysis, several challenges remain:

  • Higher-Order Generalization: The current framework is specifically tailored to third-order tensors. Extending these proofs to N-order tensors involves significantly more complex block-circulant structures.
  • The Eckart-Young Extension: While the "best T-rank one" approximation is solved, the authors leave it as an open question whether a full Eckart-Young theorem (governing best rank-$k$ approximations) holds for T-rank in the same way it does for matrices.
  • Computational Complexity: The Kilmer-Martin mapping requires multiple SVDs and DFTs. For massive datasets, such as high-resolution hyperspectral imaging, the computational overhead of maintaining sorted singular values across all Fourier slices could become a significant bottleneck.

In conclusion, by proving the invariance of singular values under orthogonal transformations, the authors justify the use of T-SVD in sensitive signal processing and machine learning applications, ensuring that the features we extract from 3D data are mathematically sound.

Summary updated

BibTeX
@article{qi2021orthogonal,
  author = {Liqun Qi and Ling Chen and Jinejie Liu and Chen Ouyang},
  title = {{Orthogonal Equivalence, Singular Values and T-Rank of Third Order Tensors.}},
  journal = {Journal of Industrial and Management Optimization},
  year = {2021},
}
Figure