The GSVD: Where are the Ellipses?, Matrix Trigonometry, and More

Alan Edelman, Yuyang Wang

TL;DR

This paper introduces a novel geometric interpretation of the Generalized Singular Value Decomposition (GSVD), framing it as a generalization of planar trigonometry to matrices. By visualizing the GSVD through "ellipse pictures," the authors aim to make this powerful decomposition more accessible and highlight its natural applications in various fields, from signal processing to data analysis.

What Problem Does This Paper Address?

The authors observe that while the Singular Value Decomposition (SVD) has a well-established and intuitive geometric interpretation (e.g., the action of a matrix on a unit ball leading to an ellipse), the Generalized Singular Value Decomposition (GSVD) lacks a similar accessible geometric understanding. This lack of intuitive visualization, they argue, contributes to the GSVD being less widely understood and utilized than it deserves. The paper aims to bridge this gap by providing a geometric framework for the GSVD, akin to the familiar geometric interpretations of the SVD.

Methods

The core of the paper's contribution lies in its novel geometric interpretation of the GSVD. The authors introduce an "ellipse picture" that visualizes the GSVD in a multi-dimensional space. This visualization involves:

  • The Red Plane: Representing the span of the combined matrices [A; B].
  • X and Y Multiaxes: These are defined as specific subspaces within the larger space, acting as projection axes.
  • Cosine and Sine Ellipses: These ellipses are projections of the unit sphere onto the X and Y multiaxes, with their semi-axes directly related to the cosine and sine components of the GSVD.

They also propose a "GH decomposition" where G represents the geometric information (orthonormal basis for the hyperplane) and H represents the coordinates in that basis. The paper further explores connections to matrix trigonometry, viewing the GSVD as a generalization of planar trigonometry, and discusses its relationship with the CS decomposition.

Datasets

Not discussed in the paper. The paper focuses on theoretical development and geometric interpretation, illustrating concepts with small dimensional examples rather than large-scale datasets.

Key Insights and Ideas

  • GSVD as Matrix Trigonometry: The paper convincingly argues that the GSVD is a natural generalization of planar trigonometry. Just as b/a represents a slope in 2D, the GSVD provides a "multi-slope" interpretation for matrices, indicating how one matrix relates to another in various directions.
  • The Ellipse Picture: The introduction of the "ellipse picture" provides a much-needed geometric intuition for the GSVD, making it more accessible by relating it to familiar SVD visualizations.
  • Natural Coordinates for Grassmann Manifold: The authors propose that the [UC; VS] part of the GSVD can serve as natural coordinates for r-dimensional hyperplanes, offering a geometric link to the Grassmann manifold.
  • De-emphasizing Eigenproblems: The paper advocates for treating the GSVD as a distinct decomposition, separate from generalized eigenvalue problems or the SVD, encouraging a more direct and natural understanding.
  • Geometric Interpretation of Regularization: The paper offers a geometric perspective on Tikhonov regularization, explaining the "calming effect" of the regularization parameter through trigonometric interpretations.

Key Results

  • Geometric Framework: The paper establishes a clear geometric framework for understanding the GSVD through its "ellipse picture" and its connection to matrix trigonometry.
  • Theoretical Connections: It proves a theorem relating the GSVD of (A, B) to the SVD of AB†, highlighting that they are not always identical, especially when infinite singular values are involved.
  • Applications: The paper demonstrates how this geometric understanding can be applied to various fields, including Tikhonov regularization, genome reconstruction, signal processing, and statistical analysis (ANOVA, discriminant analysis).
  • Software Documentation Analysis: The authors analyze the documentation of GSVD implementations in common software packages (MATLAB, Mathematica, R, Julia), suggesting that current presentations may not fully convey the GSVD's geometric richness.

Limitations and Future Work

  • Complexity of Higher Dimensions: While the ellipse picture is introduced, fully grasping the GSVD's geometry in higher dimensions can still be challenging, as acknowledged by the authors.
  • Software Presentation: The authors suggest that current software documentation for GSVD could be improved to better reflect its geometric interpretations.
  • Further Applications: The paper encourages further exploration of GSVD applications, suggesting that its geometric insights can unlock new opportunities in various scientific and engineering domains.
  • Infinite Singular Values: The paper touches upon the complexities arising from infinite singular values and their relationship with pseudoinverses, indicating potential areas for deeper theoretical investigation.

Summary updated

Abstract

This paper provides an advanced mathematical theory of the Generalized Singular Value Decomposition (GSVD) and its applications. We explore the geometry of the GSVD which provides a long sought for ellipse picture which includes a horizontal and a vertical multiaxis. We further propose that the GSVD provides natural coordinates for the Grassmann manifold. This paper proves a theorem showing how the finite generalized singular values do or do not relate to the singular values of $AB^\dagger$. We then turn to the applications arguing that this geometrical theory is natural for understanding existing applications and recognizing opportunities for new applications. In particular the generalized singular vectors play a direct and as natural a mathematical role for certain applications as the singular vectors do for the SVD. In the same way that experts on the SVD often prefer not to cast SVD problems as eigenproblems, we propose that the GSVD, often cast as a generalized eigenproblem, is rather best cast in its natural setting. We illustrate this theoretical approach and the natural multiaxes (with labels from technical domains) in the context of applications where the GSVD arises: Tikhonov regularization (unregularized vs regularization), Genome Reconstruction (humans vs yeast), Signal Processing (signal vs noise), and statistical analysis such as ANOVA and discriminant analysis (between clusters vs within clusters.) With the aid of our ellipse figure, we encourage in the future the labelling of the natural multiaxes in any GSVD problem.
BibTeX
@article{edelman2020gsvd,
  author = {Alan Edelman and Yuyang Wang},
  title = {{The GSVD: Where are the Ellipses?, Matrix Trigonometry, and More}},
  journal = {SIAM J. Matrix Anal. Appl.},
  year = {2020},
  url = {https://openreview.net/forum?id=iKFkiTeRSi},
}
Figure