Abstract
Typically flat filling, linear or polynomial interpolation methods to generate missing historical data. We introduce a novel optimal method for recreating data generated by a diffusion process. The results are then applied to recreate historical data for stocks.
Introduction
In the field of quantitative finance, the calibration of market models—the process of adjusting model parameters to match observed market prices—is a fundamental task. However, this process is frequently hindered by the "censoring" of data. Many financial assets, particularly those in illiquid markets, suffer from significant gaps in their historical price records.
Traditional approaches to this problem, such as "flat filling" (carrying the last known price forward) or linear interpolation, are deterministic. While computationally simple, these methods fail to account for the inherent volatility and stochastic nature of financial markets. They produce trajectories that are "too smooth" or "too static," leading to biased calibrations. This paper introduces a mathematically rigorous framework for "optimal stochastic decensoring," which recreates missing data in a way that is consistent with the underlying diffusion processes of the market.
The Problem of Deterministic Interpolation
The primary issue with standard interpolation methods is their disregard for the signal-to-noise ratio and the probabilistic dynamics of asset prices. If an asset is assumed to follow a Stochastic Differential Equation (SDE), any method used to fill gaps in its history should ideally produce a path that is a likely realization of that SDE.
Deterministic methods do not respect the "noise" inherent in the market. Consequently, when these "filled" datasets are used to calibrate complex models (like local or stochastic volatility models), the resulting parameters may not accurately reflect the true risk profile of the asset.
Methodology: A Two-Stage Framework
The paper proposes a solution grounded in stochastic filtering and the principle of minimal perturbation. The framework operates in two primary stages:
1. Optimal Backfilling via Time-Reversal
The first stage involves establishing a "backward" stochastic filter. If we understand the forward dynamics of an asset (the "signal") and its relationship to observable, liquid benchmarks, we can derive equations to project the conditional expectation of the price into the past.
In Theorem 2.1 (Optimal Backfilling Equations), the author derives the dynamics for this time-reversed conditional expectation. This allows researchers to use current information and liquid benchmark data to "look back" and estimate the most probable state of the illiquid asset at a previous point in time.
2. Conditional Stochastic Decensoring
While backfilling provides an expectation, "decensoring" requires the recreated path to pass exactly through known historical data points. This is analogous to a Brownian Bridge but generalized for complex diffusion processes.
To achieve this, the author utilizes Conditional SDEs. Because there are infinitely many paths that could connect two known price points, the paper employs the principle of least entropy (or minimal variance). By seeking a change of probability measure ($Q \ll P$) that minimally perturbs the original model, the author identifies a unique stochastic process that "hits" the required historical observations while remaining as faithful as possible to the "real-world" dynamics.
Practical Implementations
Recognizing that the general solution for stochastic decensoring can be computationally intensive, the paper provides two key simplifications:
- Kalman-Bucy Backfilling (Theorem 2.2): For cases where asset dynamics are linear and the noise is Gaussian, the general equations simplify into a Kalman-Bucy framework. This version is highly efficient and suitable for large-scale financial datasets.
- Relaxed Interpolation (Corollary 2.1): The author offers a "relaxed" version of the optimal path. This corollary provides a bridge between the high-level theory and practical implementation, offering a method that is easier to code than the full Theorem 2.3 while maintaining a foundation in optimal measure theory.
Key Insights
- Filtering as Reconstruction: The conceptual shift from seeing data gaps as a geometric problem to seeing them as a time-reversed filtering problem is the paper's core innovation.
- The Role of Benchmarks: The framework leverages liquid "benchmarks" (assets with continuous data) to guide the estimation of the illiquid asset. This ensures the reconstructed data is correlated with broader market movements.
- Mathematical Rigor: Unlike heuristic interpolation, this method provides a path that is a valid realization of a diffusion process, ensuring that the statistical properties of the "decensored" data match the assumed market model.
Limitations and Future Work
The author notes that the full implementation of Theorem 2.3 (Optimal Stochastic Decensoring) can be complex and may present challenges for real-time applications. Furthermore, the method is model-dependent; it assumes the underlying asset follows a continuous diffusion process. If the market experiences jumps or regime shifts not captured by the SDE, the decensoring accuracy may decrease.
Future applications of this work include the improved calibration of local volatility and stochastic volatility models in data-poor environments, where accurate historical trajectories are essential for pricing and risk management.
Conclusion
By combining time-reversal in stochastic filtering with the principle of minimal entropy perturbation, this paper provides a robust mathematical foundation for recreating missing financial data. This "optimal decensoring" ensures that historical datasets remain consistent with the stochastic nature of the markets, providing a more reliable basis for model calibration and financial analysis.
Summary updated