BSDE formulation of combined regular and singular stochastic control problems

Bruno Bouchard, Patrick Cheridito, Ying HuarXiv 2018 · DOI · arXiv

Abstract

In this paper we study a class of combined regular and singular stochastic control problems that can be expressed as constrained BSDEs. In the Markovian case, this reduces to a characterization through a PDE with gradient constraint. But the BSDE formulation makes it possible to move beyond Markovian models and consider path-dependent problems. We also provide an approximation of the original control problem with standard BSDEs that yield a characterization of approximately optimal values and controls.

Introduction

Stochastic control theory typically addresses system adjustments through two mechanisms: regular control, which involves smooth, continuous adjustments, and singular control, which accounts for sudden, cumulative impacts such as discrete jumps. Traditionally, when these problems are Markovian—meaning the future state depends only on the current state—they are solved using Partial Differential Equations (PDEs) with gradient constraints.

However, many real-world scenarios in finance and engineering are path-dependent, where the history of the process significantly influences optimal decision-making. PDEs struggle with path-dependency because they require a fixed, finite-dimensional state space. This paper addresses this limitation by reformulating the control problem using Backward Stochastic Differential Equations (BSDEs). Unlike PDEs, BSDEs are defined on the underlying probability space, making them naturally suited for settings where the entire history of the system matters.

Methodology

The authors employ a framework that bridges control theory and BSDEs through several key mathematical techniques:

Constrained BSDEs and Minimal Supersolutions

The optimal value of the control problem is represented as the "minimal supersolution" of a BSDE. In this setup, the process $Z$ (the "gains" process) is subject to specific constraints that mirror the gradient constraints found in traditional PDE approaches. Instead of seeking a unique solution, the authors identify the smallest process that satisfies a set of inequalities, which serves as the BSDE equivalent of a value function.

Weak Formulation and Girsanov’s Theorem

To handle controlled dynamics, the authors utilize a weak formulation. By applying Girsanov’s theorem, they use a change of measure to simplify the underlying process, allowing the controls to be treated as part of the "driver" of the BSDE.

The Face-lifting Technique

Singular controls can cause the system to "jump" at the terminal time $T$, potentially making the terminal reward function $h$ inconsistent with the system constraints. The authors use a "face-lifting" technique to modify the terminal condition. This pre-calculates the optimal jump at the final moment, restoring continuity to the BSDE solution and ensuring stability at the time horizon.

Key Theoretical Results

The paper provides three primary mathematical contributions:

  1. Representation Theorem (Theorem 2.2): The authors prove that the optimal value of the combined control problem is exactly equal to the initial value ($Y_0$) of the minimal supersolution of their constrained BSDE.
  2. Approximation Theorem (Theorem 2.4): They demonstrate that complex singular control problems can be approximated using a sequence of standard BSDEs. By restricting controls to piecewise constant functions, the value of the simplified problem converges to the value of the original problem as the granularity increases ($j \to \infty$).
  3. Stability (Proposition 2.5): The research shows that using the "face-lifted" terminal condition ensures the BSDE solution remains continuous at the time horizon $T$, a critical property for applications in mathematical finance.

Insights and Future Directions

The primary advantage of the BSDE formulation is the elimination of the Markovian requirement. This allows for the theoretical solution of control problems where rewards depend on moving averages or historical maxima.

While the paper provides a robust approximation bridge via standard BSDEs, it remains a theoretical work. The results rely on specific regularity conditions, such as Lipschitz continuity and boundedness of coefficients. The transition from these theoretical proofs to numerical algorithms for high-dimensional, path-dependent problems represents the next significant challenge in the field.

Summary updated

Abstract

In this paper we study a class of combined regular and singular stochastic control problems that can be expressed as constrained BSDEs. In the Markovian case, this reduces to a characterization through a PDE with gradient constraint. But the BSDE formulation makes it possible to move beyond Markovian models and consider path-dependent problems. We also provide an approximation of the original control problem with standard BSDEs that yield a characterization of approximately optimal values and controls.
BibTeX
@misc{bouchard2018bsde,
  author = {Bruno Bouchard and Patrick Cheridito and Ying Hu},
  title = {{BSDE formulation of combined regular and singular stochastic control problems}},
  year = {2018},
  eprint = {1801.03336},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1801.03336},
}
Figure