Abstract
Tensors are generalizations of matrices. They can be viewed as tables of higher dimension. A $2 \times 2$-matrix is a table that contains four numbers aligned in two directions, and each direction has dimension 2. Similarly, a $2 \times 2 \times 2$-tensor is a table with eight numbers aligned in three directions where each direction has dimension 2.
Introduction: Beyond Order-2 Tensors
In classical linear algebra, matrices (order-2 tensors) are well-understood. We possess robust tools like the Singular Value Decomposition (SVD) to find optimal low-rank approximations. Furthermore, the set of matrices with rank at most $r$ forms a well-behaved algebraic variety. However, as we transition to order-3 tensors and beyond, these familiar properties often fail to hold.
The fundamental challenge addressed in this work is the extension of metric concepts—such as distance, rank, and volume—to higher-order tensors. Central to this investigation is the Euclidean Distance (ED) problem: given an arbitrary tensor, how do we identify the closest rank-one tensor? This query leads to a deep exploration of the geometry of the Segre variety (the set of rank-one tensors) and the Veronese variety (the set of symmetric rank-one tensors).
Methods: Multilinear Algebra and Metric Geometry
The study employs a synthesis of multilinear algebra and metric algebraic geometry. Key methodological pillars include:
- Multilinear Multiplication: Generalizing the matrix operation $A M B^\top$ to higher dimensions, defining how tuples of matrices act upon a tensor.
- Optimization via Gradients: Solving the ED problem by computing the gradient of the distance function between a general tensor and the target variety.
- Projective Geometry: Utilizing complex projective space ($\mathbb{P}^N$) alongside the Kinematic Formula and the Bombieri–Weyl metric to calculate the volumes and algebraic degrees of tensor varieties.
- Polynomial Bijection: Exploiting the isomorphism between symmetric tensors and homogeneous polynomials to translate tensor problems into the language of algebraic forms.
Key Insights and the Geometry of Rank
The Ill-Posedness of Tensor Rank
A striking discovery in tensor theory is that the set of tensors with rank at most $r$ (for $r \ge 2$) is generally not closed in the Euclidean topology. It is possible for a sequence of rank-2 tensors to converge to a rank-3 tensor. Consequently, a "best rank-$r$ approximation" may not exist for certain tensors, a stark departure from matrix theory.
Identifiability
While matrix decompositions are typically not unique without additional constraints (e.g., orthogonality), higher-order tensors often possess a unique rank-$r$ decomposition. This property, known as "identifiability," is a cornerstone of modern signal processing and data science, allowing for the recovery of latent components that would be ambiguous in a matrix setting.
Singular Vectors and Eigenvectors
The "singular vector tuples" of a tensor are identified as the critical points of the distance function to the Segre variety. For symmetric tensors, this leads to the definition of tensor eigenvectors. A general tensor of order $d$ and dimension $n$ has a finite number of complex eigenvectors, determined by the formula $\sum_{i=0}^{n-1} (d-1)^i$.
Key Results
- ED Degree of the Segre Variety: The research provides a formula for the number of singular vector tuples for a general tensor. For a $2 \times 2 \times 2$ tensor, this "ED degree" is 6.
- Volume and Degree Formulas: Explicit formulas are derived for the volumes and degrees of the Segre and Veronese varieties. For instance, the degree of the Veronese variety $\mathcal{V}_{n,d}$ is $d^{n-1}$.
- Non-Closure Proof: The text offers a constructive proof demonstrating how rank-2 tensors can limit to a rank-3 tensor, highlighting the inherent complexity of tensor approximation.
Conclusion and Future Directions
While the rank-one case is well-understood through singular vectors, the problem of finding the best rank-$r$ approximation for $r > 1$ remains difficult due to the non-closure of rank sets. Future research continues to explore the geometry of Segre–Veronese varieties and the statistical properties of real eigenvectors, moving beyond the cleaner results found in complex projective space.
Summary updated