When evaluating how quickly an approximation converges to a true solution, our first instinct is to ask how fast the error shrinks to zero. However, a critical and often overlooked variable is how we choose to measure that error. Choosing an error metric is not just a matter of mathematical bookkeeping; it fundamentally dictates the speed limit of convergence.
Across numerical analysis, statistical inference, and partial differential equations, a universal principle emerges: there is an inherent trade-off between the geometric or structural smoothness of the norm used to measure error and the resulting rate of convergence. The more demanding we are about the regularity of our error metric, the slower our approximation converges.
The Intuition: Why Smoothness Penalizes Speed
To understand why this trade-off exists, imagine approximating a smooth target function with a sequence of numerical approximations that contain high-frequency, jagged oscillations.
If we measure the error using a weak norm (such as a standard pointwise distance or an $L^2$ space), the metric only cares about the average or local magnitude of the discrepancy. As long as the peaks of the oscillations stay close to the true curve, the error drops quickly.
Weak Norm (e.g., L²): Cares only about distance.
Approximation: /\/\/\/\/\ <- Oscillations are fine if they are small.
Target: ___________
Strong Norm (e.g., Hölder): Penalizes the sharpness of the slopes.
Approximation: /\/\/\/\/\ <- High penalty due to steep derivatives!
If we switch to a strong norm—such as a Hölder or Sobolev norm—the metric penalizes not just the distance from the target, but also the derivatives or pathwise roughness of the error. For the error to become small in a strong norm, the approximation must completely smooth out its oscillations. Forcing this structural regularity requires significantly more data, tighter domain constraints, or more computational steps, which drags down the overall convergence rate.
The Exact Cost of Pathwise Regularity
This rate reduction is not a vague qualitative phenomenon; it can be quantified precisely. The work of Sonja Cox, Arnulf Jentzen, Martin Hutzenthaler, Jan van Neerven, and Timo Welti in Convergence in Hölder norms with applications to Monte Carlo methods in infinite dimensions demonstrates this exact mathematical tax in infinite-dimensional spaces.
Consider a numerical scheme approximating a stochastic process. If the scheme converges at a base rate of $\beta$ under a standard norm, its pathwise convergence in a stronger Hölder norm with exponent $\alpha$ is reduced to:
$$\text{Convergence Rate} = \beta - \alpha$$
The Hölder exponent $\alpha$ acts as a direct subtraction from the baseline speed. To guarantee that an approximation captures the fine, pathwise smoothness of a stochastic process, one must accept a strictly slower rate of convergence.
Statistical Implications in Bayesian Inversion
This trade-off also governs how statistical models learn from physical data. In Bayesian inverse problems, the goal is to reconstruct an unobserved cause from noisy observations by calculating a posterior probability distribution.
In Posterior consistency and convergence rates for Bayesian inversion with hypoelliptic operators, Kekkonen et al. explore how this rate-smoothness trade-off manifests as the data size grows. When evaluating how fast the posterior distribution contracts around the true solution, the authors show that choosing a stronger, smoother norm to measure error slows down the contraction rate. Conversely, weaker norms allow for faster contraction toward the true solution, illustrating that the penalty for demanding structural regularity is universal across both deterministic and statistical frameworks.
Domain and Operator Regularity Constraints
The relationship between regularity and convergence speed extends beyond the choice of error metrics to the intrinsic geometric and structural properties of the problem itself.
In State-Constraint Static Hamilton-Jacobi Equations in Nested Domains, Kim et al. analyze how solutions to partial differential equations behave as their domains expand. They show that the regularity and convexity of the governing operator (the Hamiltonian) dictate the convergence rate. A highly regular, convex Hamiltonian allows for rapid, exponential convergence. However, when the operator is nonconvex, the lack of structural regularity introduces complex solution behaviors, forcing the convergence rate down to a slower polynomial speed.
This geometric dependency is mirrored in the work of Wolfgang zu Castell and Frank Filbir on Cesàro means of Jacobi expansions on the parabolic biangle. The authors establish that the specific boundary characteristics of a domain fundamentally alter the uniform convergence conditions of orthogonal expansions. Moving away from simple geometries (like squares or balls) to complex domains changes the summability behavior, proving that convergence rates cannot be decoupled from spatial and geometric regularity.
Summary
Whether you are bounding the paths of a stochastic differential equation, contracting a Bayesian posterior, or expanding the domain of a Hamilton-Jacobi equation, the mathematical reality is the same. Speed and smoothness are competing forces. High convergence rates require flexible, weak metrics, while strong, regular metrics demand a sacrifice in speed. Understanding this trade-off allows researchers to choose the exact balance of norm strength and convergence speed required for their specific application.
Key Foundational Work
Cesàro means of Jacobi expansions on the parabolic biangle (2009) — contributed 2 concept elements to this direction.
Posterior consistency and convergence rates for Bayesian inversion with hypoelliptic operators (2016) — contributed 1 concept elements to this direction.