When modeling complex, non-linear systems, our first instinct is often to reach for polynomials. Whether we are approximating a smooth function, solving a differential equation, or relaxing a non-convex optimization problem into a convex one, polynomials serve as the fundamental mathematical scaffolding.
However, a severe hidden trap emerges as we push these models to higher degrees: numerical ill-conditioning. In floating-point arithmetic, the way we mathematically represent a polynomial can mean the difference between a fast, machine-precision solution and total algorithmic divergence.
A major research direction focuses on solving this crisis by shifting from standard monomial bases to optimized, orthogonal polynomial bases.
The Root of the Problem: Monomial Collapse
To understand why standard polynomial representations fail, consider the standard monomial basis: $1, x, x^2, x^3, \dots$.
Imagine plotting these functions on the interval $[0, 1]$. As the exponent $n$ grows, the functions $x^{10}$, $x^{11}$, and $x^{12}$ begin to look virtually identical. They are all nearly flat near zero, and they all shoot up sharply to $1$ at the boundary.
Monomial Basis (Ill-Conditioned) Orthogonal Basis (Well-Conditioned)
Value Value
^ ^
1 | / / / 1 | /\ /\ /
| / / / | / \ / \ /
| / / / | / \/ \/
0 +------------> 0 +------------>
0 1 X -1 1 X
(Functions become collinear) (Functions remain distinct)
When we construct matrices using these monomials—such as Vandermonde matrices in regression or moment matrices in semidefinite programming—the columns become nearly collinear. As Haim Avron et al. point out in Sketching Structured Matrices for Faster Nonlinear Regression, this reliance on monomial bases inevitably results in ill-conditioned structures.
In floating-point arithmetic, an ill-conditioned matrix means that tiny rounding errors amplify exponentially. Because a computer cannot easily distinguish between the contributions of $x^{10}$ and $x^{12}$, optimization algorithms stall, coefficients fluctuate wildly, and solvers ultimately fail to converge.
The Mechanism: Geometric Decoupling via Orthogonality
The core insight of this research direction is that we can change the coordinate system of our function space. Instead of using monomials that crowd together, we can represent polynomials using an orthogonal basis—such as Chebyshev, Legendre, or Bernstein polynomials.
Geometrically, two functions are orthogonal if their inner product over a specific domain is zero:
$$\int T_i(x) T_j(x) w(x) dx = 0 \quad \text{for } i \neq j$$
This algebraic property has a profound physical effect on computation. Instead of mimicking each other, orthogonal polynomials oscillate evenly across the domain. Each higher-degree term adds a completely new, independent geometric wave that cannot be replicated by lower-degree terms.
By switching to an orthogonal basis, the off-diagonal elements in our data matrices drop significantly, effectively decoupling the underlying equations. As Alex Townsend et al. demonstrate in A fast and well-conditioned spectral method, using specific degree-graded polynomial bases ensures that the condition number of the resulting linear system remains bounded. This allows practitioners to solve high-order differential equations with millions of variables without encountering exponential error growth.
Rescuing Semidefinite Programming and Moment Hierarchies
This numerical breakdown is particularly destructive in Sum-of-Squares (SOS) optimization and the Lasserre moment hierarchy. These frameworks translate notoriously difficult, non-convex polynomial problems into convex Semidefinite Programs (SDPs). However, as Giovanni Fantuzzi et al. note in Polynomial Optimization for Nonlinear Dynamics: Theory, Algorithms and Applications, high-degree polynomial representations create severely ill-conditioned matrices that disrupt standard SDP solvers.
When calculating integrals or volumes over complex shapes, Jean B. Lasserre et al. show in Approximate Volume and Integration for Basic Semialgebraic Sets that replacing the standard power basis with Chebyshev polynomials dramatically mitigates this ill-conditioning. The orthogonal representation prevents the exponential growth of coefficients, allowing the solver to achieve higher-degree approximations and better precision.
Similarly, in inverse moment problems, Didier Henrion et al. establish in Mean Squared Error Minimization for Inverse Moment Problems that orthonormal bases like Legendre or Chebyshev polynomials are strictly necessary to maintain stability. By decorrelating the errors between different moments, these bases allow solvers to reliably reconstruct probability densities where standard monomial approaches fail due to floating-point errors.
Ubiquitous Stability Across Domains
The paradigm of basis optimization extends across a wide variety of computational fields:
- Reinforcement Learning: In continuous state spaces, Davide Maran et al. demonstrate in No-Regret Reinforcement Learning in Smooth MDPs that standard polynomial features fail to achieve meaningful policy learning due to numerical instability. Transitioning to orthogonal Legendre polynomials provides the numerical stability required to accurately learn environment dynamics.
- Computer Vision and Shape Reconstruction: When reconstructing shapes from image moments, Costanza Conti et al. show in Stable recovery of planar regions with algebraic boundaries in Bernstein form that switching from a power basis to a Bernstein basis prevents coefficients from fluctuating wildly in the presence of noise, enabling robust recovery from low-resolution samples.
- Deep Learning Acceleration: In hardware-accelerated convolutions, Kevin Vincent et al. demonstrate in On Improving the Numerical Stability of Winograd Convolutions that carefully selecting polynomial evaluation points mitigates the exponential growth of values in underlying Vandermonde matrices, preserving numerical precision for larger tile sizes.
Conclusion
Numerical instability is often treated as an unavoidable tax on high-degree polynomial approximations. However, this research direction proves that instability is frequently an artifact of how we represent functions rather than the functions themselves. By shifting from collinear monomials to orthogonal bases, researchers across disciplines isolate and decouple numerical errors, unlocking high-precision computation for large-scale, complex systems.
Key Foundational Work
GloptiPoly: Global optimization over polynomials with Matlab and SeDuMi (2003) — contributed 1 concept elements to this direction.