A martingale analysis of first passage times of time-dependent Wiener diffusion models

Vaibhav Srivastava, Samuel F. Feng, Jonathan D. Cohen, Naomi Ehrich Leonard, Amitai ShenhavJournal of Mathematical Psychology 2016 · DOI · arXiv

Abstract

This paper introduces a rigorous mathematical framework for analyzing decision-making processes where evidence accumulation parameters change over time. By applying martingale theory and the Optional Stopping Theorem to Wiener diffusion models, the authors derive closed-form analytic solutions for first passage time (FPT) densities, choice probabilities, and mean response times in multistage decision tasks. This approach provides an efficient alternative to computationally intensive numerical simulations and extends the classical Drift Diffusion Model (DDM) to accommodate dynamic environments, such as those involving shifting attention or increasing urgency.

Introduction

In psychology and neuroscience, the Drift Diffusion Model (DDM) is considered the "gold standard" for modeling two-choice decisions. It conceptualizes the decision process as a particle (representing evidence) drifting toward one of two boundaries (representing choices). Traditionally, these models assume that the drift rate (signal strength) and diffusion rate (noise) remain constant throughout the decision process.

However, real-world decisions are rarely static. Information may arrive in bursts, or a decision-maker might become increasingly "urgent" as a deadline approaches, leading to collapsing decision thresholds. While "two-stage" models have existed for decades, generalizing them to an arbitrary number of stages has been mathematically challenging, often requiring researchers to solve complex partial differential equations or rely on slow Monte-Carlo simulations. This paper provides an elegant, analytic solution to these multistage problems.

Methodology

The Martingale Approach

Rather than solving the Kolmogorov backward equations—the standard calculus-heavy approach—the authors leverage Martingale Theory. In probability theory, a martingale is a model of a "fair game" where the future expectation of a variable is equal to its current value. The researchers identified specific functions of the diffusion process that satisfy the martingale property.

Optional Stopping Theorem (OST)

The core of the analytic derivation is the Optional Stopping Theorem. The OST states that for a martingale, the expected value at a random stopping time (the moment a decision boundary is reached) is equal to its initial value. This allows the authors to bypass the complex path of the accumulation process and jump directly to the statistics of the "finish line."

Multistage Recursion

To handle time-varying parameters, the authors partition the decision timeline into $n$ discrete intervals. They treat the end of one stage as the starting point for the next. By conditioning the state of the evidence at time $t_i$ on the fact that no decision was reached in any previous stage, they chain together multiple stages into a single analytic framework. This "piecewise constant" approach allows for the modeling of complex dynamics, such as alternating drift rates or time-dependent boundaries.

Ornstein-Uhlenbeck (O-U) Transformation

The paper also demonstrates how these methods apply to Ornstein-Uhlenbeck processes, which model "leaky" evidence accumulation. By transforming the O-U process into a time-scaled Wiener process, the authors extend their martingale framework to models with decay or reinforcement.

Validation and Key Insights

Simulation Results

The researchers validated their analytic formulas using Monte-Carlo simulations (ranging from 1,000 to 10,000 runs) using the Euler-Maruyama method. In all tested scenarios—including alternating drift (simulating shifts in attention) and collapsing thresholds (simulating urgency)—the analytic formulas matched the simulation "ground truth" with high precision.

Approximating Continuity

While the model is fundamentally piecewise, the authors demonstrate that a model with a sufficient number of stages (e.g., 20 stages) can almost perfectly approximate a process where parameters change continuously. This provides a computationally efficient way to model smooth transitions in information quality or decision criteria.

Non-Linear Reward Rates

One of the more striking insights involves the relationship between decision thresholds and reward rates in two-stage models. The authors found that this relationship can be bimodal, suggesting that small changes in the environment could lead to "discontinuous jumps" in what constitutes an optimal decision strategy.

Conclusion and Software Contribution

The primary contribution of this work is the shift from numerical approximation to algebraic exactness for complex diffusion models. To support the research community, the authors released the multistage MATLAB toolbox, which implements these formulas and allows for the reproduction of their results.

While the current framework focuses on the "forward problem" (calculating outcomes from known parameters), it establishes the mathematical foundation necessary for the "inverse problem"—fitting multistage parameters to empirical behavioral data. Future work will likely explore the computational limits of these approximations in high-frequency fluctuation scenarios and the application of these models to real-world human and animal decision-making datasets.

Summary updated

BibTeX
@article{srivastava2016martingale,
  author = {Vaibhav Srivastava and Samuel F. Feng and Jonathan D. Cohen and Naomi Ehrich Leonard and Amitai Shenhav},
  title = {{A martingale analysis of first passage times of time-dependent Wiener diffusion models}},
  journal = {Journal of Mathematical Psychology},
  year = {2016},
  doi = {10.1016/j.jmp.2016.10.001},
}

Loading related content...

Figure