TL;DR
This paper extends the "principal-agent hypothesis testing" framework from single-shot to sequential settings, allowing for adaptive experimentation and interim decision-making. The authors prove that e-processes are the fundamental tools for designing contracts that prevent "null" agents (those with ineffective treatments or products) from profiting, while providing practical, near-optimal strategies for agents to maximize their rewards.
What Problem Does This Paper Address?
In many strategic interactions, a principal (like the FDA) controls access to a market, and an agent (like a pharma company) pays a cost to prove their product is effective. Traditional statistical contracts assume a "single-shot" setup: the agent submits one dataset, and the principal decides the reward.
However, real-world experimentation is rarely single-shot. Clinical trials run for years, and tech companies run A/B tests where data accumulates over time. This introduces a major challenge: strategic stopping. If an agent can choose when to stop an experiment based on the results so far, they might "cherry-pick" a moment when the evidence looks favorable by chance, even if their product is ineffective. This paper develops a mathematical framework for sequential contracts that remain robust against such strategic behavior while allowing for flexible, adaptive experimentation.
Methods
The authors introduce two types of sequential reward schedules:
- Deferred Rewards: The agent gathers evidence over time, but the reward is only paid out once, at a strategically chosen stopping time.
- Incremental Rewards: Rewards are disbursed along the way as evidence accumulates. Because these cannot be "clawed back," the cumulative reward must be non-decreasing.
To ensure these contracts are null-deterring (meaning a "null agent" cannot expect to make a profit), the authors leverage e-processes. An e-process is a sequence of non-negative random variables whose expectation is at most 1 at any stopping time.
For incremental rewards, they utilize adjusters—mathematical functions that transform volatile evidence into a smooth, non-decreasing reward stream.
On the agent's side, the authors identify the optimal response using the Snell envelope (for stopping) and Bellman equations (for parameter selection). Recognizing that these are often computationally heavy, they propose two simpler approximations:
- CGF-optimal rule: Based on the Cumulant Generating Function of the data.
- Growth-optimal (GRO) rule: A first-order approximation that is easier to estimate from data using empirical averages.
Datasets
The paper evaluates its theoretical findings using numerical simulations based on:
- Bernoulli observations: Modeling success/failure outcomes (e.g., a drug working or not).
- Gaussian observations: Modeling continuous outcomes with a known variance.
These synthetic benchmarks allow the authors to compare their proposed practical strategies against the theoretical "Bellman benchmark" (the absolute best an agent could do with perfect knowledge).
Key Insights and Ideas
- E-processes are the "Gold Standard": Just as e-values characterize optimal single-shot contracts, e-processes are the natural language for sequential contracts. They inherently handle the "optional stopping" problem that plagues p-values.
- The Power of Adjusters: Adjusters (shown in Figure 1) act as a "budgeting" tool for the principal. They allow the principal to decide whether to reward early milestones or save the bulk of the reward for high-evidence breakthroughs.
- Robustness of Simple Rules: One of the most encouraging findings is that agents don't need to solve complex dynamic programming problems to perform well. The CGF-optimal rule typically achieves over 98% of the maximum possible utility.
- Asymmetric Guarantees: The "null-deterring" property is a powerful guarantee for the principal. It holds regardless of the agent's utility function or stopping strategy, prioritizing the integrity of the statistical test over the agent's specific goals.
Key Results
- Characterization: The paper proves that all null-deterring deferred contracts are characterized by e-processes, and provides a construction for incremental contracts using adjusters.
- Efficiency: In simulations (Figure 2), the CGF-optimal and GRO rules are shown to be highly efficient, especially for "patient" agents (those with a high discount factor $\delta$).
- Learning on the Fly: The authors demonstrate a "predictable plug-in" strategy where the agent estimates the drift of their data in real-time. While this has a "cost of learning," it converges toward the optimal strategy as more data is collected.
Limitations and Future Work
- I.I.D. Assumption: The practical approximations (CGF and GRO) currently assume that data increments are independent and identically distributed. Extending these to non-i.i.d. settings (like time-series data) is a natural next step.
- Overshoot Costs: The paper notes that "overshoot"—when a single data point pushes the evidence far past a reward threshold—can lead to utility loss for the agent. Proposition 4 quantifies this, but further work could refine strategies to mitigate it.
- Fixed Entry Costs: The model focuses on a single upfront entry cost $C$. Future research could integrate per-round experimentation costs into the e-process framework to better model scenarios where data collection is prohibitively expensive.
Summary updated