Non-Negative Stiefel Approximating Flow: Orthogonalish Matrix Optimization for Interpretable Embeddings

Brian B. Avants, Nicholas J. Tustison, James R. StonearXiv 2025 · DOI · arXiv

Abstract

The Non-Negative Stiefel Approximating Flow (NSA-Flow) is a novel matrix optimization framework designed to achieve interpretable, sparse, and "orthogonalish" data representations. By utilizing a "soft-retraction" flow to balance reconstruction accuracy with manifold constraints, NSA-Flow provides a tunable mechanism to navigate the trade-off between data fidelity and feature separation. This approach demonstrates superior performance in genomics and neuroimaging tasks compared to traditional methods like PCA and NMF.

Introduction: The Search for Interpretable Latent Factors

In high-dimensional fields such as genomics and neuroimaging, identifying latent factors—underlying patterns that explain observations—is essential. Ideally, these factors should be interpretable: sparse (involving few variables) and disjoint (minimally overlapping). However, standard tools present significant trade-offs. Principal Component Analysis (PCA) yields orthogonal components that often lack physical meaning due to negative values. Non-negative Matrix Factorization (NMF) ensures parts-based representations but suffers from "rotational ambiguity," where factors become entangled. While Riemannian optimization can enforce strict orthogonality, it is often computationally prohibitive and overly rigid for real-world data fitting.

The Non-Negative Stiefel Approximating Flow (NSA-Flow) addresses these challenges by offering a middle ground: an efficient framework that handles non-negativity while allowing for "soft" or partial orthogonality.

The NSA-Flow Framework

The core innovation of NSA-Flow is the Soft-Retraction Method. Rather than strictly projecting onto the Stiefel manifold (the space of orthonormal matrices) at every iteration, the algorithm defines a "flow" that pulls the solution toward it. This is achieved through a composite energy function: $$E(Y) = (1-w) \cdot \text{Fidelity} + w \cdot \text{Orthogonality}$$

The update rule utilizes a convex combination: $$Y^{(t+1)} = (1-w)\tilde{Y}^{(t+1)} + wQ^{(t+1)}$$ where $Q$ represents the polar retraction (the nearest point on the Stiefel manifold). The parameter $w \in [0, 1]$ serves as a "knob" to tune the trade-off between standard gradient descent and strict manifold constraints.

Flowchart of the NSA-Flow algorithm, illustrating the iterative process of gradient descent, tangent space projection, and soft-retraction.

Computational Efficiency and Method Comparison

NSA-Flow is designed for practical integration into modern deep learning pipelines. By utilizing soft-retraction interpolation instead of full SVD or matrix square root inversions at every step, the method maintains a computational complexity of $O(pk^2)$. This makes it significantly more scalable than methods like the Cayley Transform for large dimensions.

Comparison of NSA-Flow with other optimization methods, highlighting its balanced complexity and controlled orthogonality.

Experimental Results

The authors validated NSA-Flow across several benchmarks:

  • Genomics (Golub Leukemia Dataset): NSA-Flow achieved a cross-validation accuracy of 88.3%, surpassing PCA (81.9%) and Sparse PCA (86.4%).
  • Neuroimaging (ADNI Cortical Thickness): In Alzheimer’s disease classification, NSA-Flow-derived networks provided a 4.6% absolute AUC improvement over PCA.
  • Interpretability: Qualitative analysis via heatmaps revealed that NSA-Flow identifies biologically plausible regions, such as the medial temporal lobe, which are often obscured in standard PCA loadings.

Key Insights

A significant finding is that NSA-Flow induces Sparsity as a Side Effect. By enforcing near-orthogonality alongside non-negativity, columns are naturally pushed toward disjoint supports to minimize overlap, eliminating the need for explicit $L_1$ penalties. Furthermore, "orthogonalish" representations often generalize better than strictly orthogonal ones, as they avoid over-fitting to rigid manifold constraints.

Conclusion and Future Directions

NSA-Flow provides a flexible, efficient tool for discovering interpretable structures in complex data. While currently limited by the $O(k^3)$ cost of matrix inversions for very high numbers of components ($k$), future work aims to incorporate sparse matrix support and stochastic optimization. Extensions to multi-modal data fusion and second-order optimization are also being explored to further enhance the framework's utility.

Summary updated

Abstract

Interpretable representation learning is a central challenge in modern machine learning, particularly in high-dimensional settings such as neuroimaging, genomics, and text analysis. Current methods often struggle to balance the competing demands of interpretability and model flexibility, limiting their effectiveness in extracting meaningful insights from complex data. We introduce Non-negative Stiefel Approximating Flow (NSA-Flow), a general-purpose matrix estimation framework that unifies ideas from sparse matrix factorization, orthogonalization, and constrained manifold learning. NSA-Flow enforces structured sparsity through a continuous balance between reconstruction fidelity and column-wise decorrelation, parameterized by a single tunable weight. The method operates as a smooth flow near the Stiefel manifold with proximal updates for non-negativity and adaptive gradient control, yielding representations that are simultaneously sparse, stable, and interpretable. Unlike classical regularization schemes, NSA-Flow provides an intuitive geometric mechanism for manipulating sparsity at the level of global structure while simplifying latent features. We demonstrate that the NSA-Flow objective can be optimized smoothly and integrates seamlessly with existing pipelines for dimensionality reduction while improving interpretability and generalization in both simulated and real biomedical data. Empirical validation on the Golub leukemia dataset and in Alzheimer's disease demonstrate that the NSA-Flow constraints can maintain or improve performance over related methods with little additional methodological effort. NSA-Flow offers a scalable, general-purpose tool for interpretable ML, applicable across data science domains.
BibTeX
@misc{avants2025non,
  author = {Brian B. Avants and Nicholas J. Tustison and James R. Stone},
  title = {{Non-Negative Stiefel Approximating Flow: Orthogonalish Matrix Optimization for Interpretable Embeddings}},
  year = {2025},
  eprint = {2511.06425},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.06425},
}
Figure