Abstract
Greedy kernel approximation algorithms are successful techniques for sparse and accurate data-based modelling and function approximation. Based on a recent idea of stabilization of such algorithms in the scalar output case, we here consider the vectorial extension built on VKOGA. We introduce the so-called $\gamma$-restricted VKOGA, comment on analytical properties and present numerical evaluation on data from a clinically relevant application: the modelling of the human spine. The experiments show that the new stabilized algorithms result in improved accuracy and stability over the non-stabilized algorithms.
The Challenge of Numerical Instability
Kernel methods are powerful tools for approximating complex functions from scattered data, but they face a major hurdle: numerical instability. When using "greedy" algorithms to build these models—where the algorithm iteratively picks the best data points to include—the points often start to cluster together. In mathematical terms, this causes the kernel matrix to become "ill-conditioned," leading to massive numerical errors or algorithm crashes.
To avoid this, researchers typically have to stop the algorithm early, which leaves significant potential accuracy on the table. This paper addresses this by extending a "stabilization" technique, previously limited to single-output scalar functions, to vectorial functions (multi-output), specifically for use in biomechanical simulations where speed and precision are both critical.
Methods: The $\gamma$-Restricted VKOGA
The authors introduce the $\gamma$-restricted VKOGA. The algorithm builds a model by picking points one by one using standard criteria like $f$-greedy (picking where error is highest) or $P$-greedy (picking where uncertainty is highest).
The core innovation is the introduction of a restriction parameter, $\gamma \in (0, 1]$. Instead of searching the entire dataset for the next best point, the algorithm only considers a "restricted set" of points where the Power function (a measure of local approximation quality) is sufficiently large. By forcing the algorithm to pick points that maintain a minimum "separation distance," the underlying mathematics remains stable. This allows the algorithm to run for more iterations and incorporate more points without the matrix "blowing up."
Application: Human Spine Modelling
The method was evaluated on a clinically relevant biomechanical dataset consisting of 1,370 input-output pairs. The goal was to map 3D displacements (inputs) to 3D forces and rotational moments (outputs) within an intervertebral disc (IVD).
This data is typically generated by slow, computationally expensive Finite Element (FE) models. The $\gamma$-restricted VKOGA aims to create a "surrogate" that is just as accurate but significantly faster for real-time simulation.
Key Results and Performance Gains
The results show a dramatic improvement when using stabilization, particularly for the $f/P$-greedy selection method. Stabilization acts as a bridge; at $\gamma=0$, the algorithm is a standard accuracy-seeking tool, while at $\gamma=1$, it is purely geometry-based. The "sweet spot" found via cross-validation provides the best of both worlds.
- Error Reduction: For the $f/P$-greedy method, the Relative Root Mean Square Error (RMSE) dropped from 14.83 (unstabilized) to 0.0797 (stabilized).
- Model Complexity: The stabilized model was able to use 358 points before hitting stability limits, compared to only 63 points for the unstabilized version.
- Max Error: The maximum relative error ($E_{max,rel}$) for the $f$-greedy method saw a massive improvement, dropping from 6.95 to 0.179.
Conclusion and Future Work
By restricting where the algorithm can pick points, the authors paradoxically achieved a better model because the stabilized algorithm can safely incorporate hundreds of points that would otherwise cause numerical failure. While the method requires tuning hyperparameters like the stabilization parameter $\gamma$, the orders-of-magnitude improvement in reliability makes it a robust choice for multi-dimensional engineering problems. Future work will focus on integrating stabilization and regularization more deeply into the selection step and testing the method on more complex, patient-specific data.
Summary updated