Numerical methods for differential equations are typically designed to achieve a specific "order" of accuracy. A fourth-order method, for instance, suggests that halving the step size $h$ should reduce the error by a factor of $2^4 = 16$. However, when these methods are applied to stiff equations—systems characterized by rapidly decaying components or disparate timescales—this theoretical accuracy often evaporates. This phenomenon, known as order reduction, causes sophisticated high-order integrators to perform no better than much simpler, lower-order schemes.
The Breakdown of Taylor Series Analysis
The classical way to determine the order of a numerical method is through Taylor series expansion. We assume the solution is sufficiently smooth and that the error can be expressed as a power series in the step size $h$. As Kozlov et al. et al. demonstrate in The behaviour of the local error in splitting methods applied to stiff problems, this analysis is fundamentally misleading for stiff problems.
In a stiff system, there is often a hidden small parameter $\epsilon$ representing a fast relaxation time. While classical analysis assumes $h$ is the only small parameter, the actual error depends on the ratio $h/\epsilon$. When $h$ is large relative to $\epsilon$, the higher-order derivatives in the Taylor expansion—which classical theory assumes are bounded—actually explode. Consequently, the local error may stop decreasing as $h$ gets smaller, creating a characteristic "plateau" in the convergence plot where the method fails to improve until $h$ is extremely small.
Smoothness and Singularities
Order reduction is not limited to stiffness in the sense of large eigenvalues; it also occurs in singular initial value problems. As de Hoog et al. et al. explain in The application of Runge-Kutta schemes to singular initial value problems, standard explicit Runge-Kutta schemes often degrade to just second-order convergence when faced with singularities (such as those arising in cylindrical or spherical coordinate systems).
The intuition here is similar: the mathematical proof for high-order accuracy relies on the existence of bounded derivatives at the starting point. When a singularity is present, these derivatives are no longer bounded, violating the core assumptions of the scheme and imposing a "performance ceiling" that persists regardless of how many stages the method has.
The Limits of Splitting and IMEX Methods
To handle stiffness, researchers often use Implicit-Explicit (IMEX) methods or operator splitting. These techniques partition a problem into a stiff part (solved implicitly or exactly) and a non-stiff part (solved explicitly).
However, even these specialized methods are not immune to accuracy degradation:
- Operator Splitting: Tapley et al. et al. show in A novel approach to rigid spheroid models in viscous flows using operator splitting methods that second-order splitting methods can drop to first-order accuracy when dissipative forces dominate. This occurs because the coupling between the split parts becomes the dominant source of error, and this coupling error does not vanish at the expected rate in the stiff regime.
- IMEX Methods: As noted by Zhang et al. et al. in High Order Implicit-explicit General Linear Methods with Optimized Stability Regions, high-order IMEX Runge-Kutta methods frequently suffer from order reduction. The stiff components, even when treated implicitly, can "pollute" the overall accuracy, forcing the user to choose between very small time steps or lower-than-expected precision.
Recovering High Order
Overcoming order reduction requires moving beyond standard Taylor-based constructions. One approach involves exponential integrators, which use the matrix exponential to solve the linear stiff part exactly. However, even these have trade-offs. Owren et al. et al. found in Solving the nonlinear Schrödinger equation using exponential integrators that different classes of exponential integrators are sensitive to different smoothness constraints—some are bottlenecked by the physical potential, while others are limited by the initial conditions.
More recently, researchers have looked toward polynomial block methods. In Additive Polynomial Time Integrators, Part I: Framework and Fully Implicit-Explicit Collocation Methods, Buvoli and Southworth et al. propose a framework using Radau collocation to eliminate order reduction entirely. By constructing the integrator as a single polynomial block rather than a sequence of discrete stages, these methods can maintain their theoretical high-order accuracy even in the presence of extreme stiffness, while simultaneously enabling parallel computation.
Key Foundational Work
The behaviour of the local error in splitting methods applied to stiff problems (2003) — contributed 4 concept elements to this direction.
The application of Runge-Kutta schemes to singular initial value problems (1985) — contributed 3 concept elements to this direction.