Towards a Theory of Confidence in Market-Based Predictions

Rupert Freeman, David M. Pennock, Daniel M. Reeves, David M. Rothschild, Bo WaggonerISIPTA 2021

Abstract

Prediction markets are widely regarded as effective tools for aggregating information into a single probability estimate. However, unlike traditional statistical models, they lack a standardized measure of confidence or a "margin of error." This paper proposes two formal frameworks to quantify market confidence: an efficiency-based measure derived from market frictions and an information-based measure derived from expected price volatility. By framing confidence through the lens of potential profit and second-order probabilities, the authors provide a theoretical foundation for more nuanced market-based forecasting.

Introduction

In a prediction market, the price of a contract is typically interpreted as the probability of an event occurring. For instance, a contract trading at $0.70 implies a 70% probability. While this provides a clear point estimate, it fails to convey the "hardness" of the prediction. A 70% forecast can represent two very different states of knowledge: a stable estimate based on exhaustive historical data, or a "soft" placeholder in a market awaiting a major news event that will likely swing the odds to either 90% or 50%.

Standard statistical forecasts often include a margin of error or a confidence interval to signal the reliability of the data. Prediction markets currently lack this secondary metric. This paper seeks to bridge that gap by asking: How can we formalize the "feeling" of confidence in a market-based prediction? The authors aim to determine when a market probability meaningfully reflects underlying certainty versus when it is merely a reflection of missing information or market friction.

Proposed Frameworks for Measuring Confidence

The authors propose two distinct, complementary frameworks for measuring market confidence: one based on market efficiency and the other on information flow.

1. The $x$ Margin of Error (Efficiency-Based)

This approach leverages the Efficient Market Hypothesis (EMH). In a perfectly efficient market with zero fees and infinite liquidity, any mispricing would be immediately corrected by rational traders. However, real-world markets contain frictions such as trading fees, low liquidity, and investment caps that allow small errors to persist because they are not profitable to correct.

To define confidence, the authors introduce the concept of an "omniscient trader" who knows the true probability $p$. The "$x$ margin of error" is defined as the set of all probabilities that would allow this omniscient trader to make a profit of at most $x$. In a "thick" market with high liquidity and low fees, this margin is narrow, indicating high confidence. In a "thin" market where frictions are high, the margin of error widens, signaling that the market price may deviate significantly from the true probability without being corrected.

2. Expected Volatility (Information-Based)

This approach focuses on second-order probabilities—the probability that the current probability estimate will change. This is particularly relevant when the market anticipates new information. For example, if a major announcement is expected in 24 hours, the current market price is likely to be highly volatile.

The authors suggest that expected volatility can serve as a proxy for confidence regarding future information. They propose using financial derivatives, such as butterfly options (which pay out based on the difference between prices at two different times), to quantify this expectation. High expected volatility implies low confidence in the current price as a stable, long-term forecast, even if the market is currently efficient.

Key Insights

  • Market Frictions as Confidence Bounds: Trading fees and liquidity constraints are not merely inefficiencies; they provide a mathematical basis for bounding uncertainty. If a market has a 5% fee, the price is only "reliable" within a 5% margin, as any error smaller than that is economically invisible to traders.
  • The Multi-Dimensionality of Confidence: The paper argues that confidence is not a monolithic concept. A market can have a narrow margin of error (reflecting all current information accurately) while simultaneously having high expected volatility (indicating that new information will soon change the price).
  • The Omniscient Trader Bridge: By framing confidence through potential profit, the authors create a theoretical bridge between abstract probability theory and the practical mechanics of financial markets.

Conclusion and Future Directions

As a theoretical framework, this paper provides a conceptual way to report market forecasts with greater nuance—for example, reporting a probability as "65%, with a $1,000 margin of error between 57% and 70%."

However, several challenges remain for future research. The authors acknowledge the need for empirical validation to determine if "high-confidence" market predictions actually correlate with higher accuracy (lower Brier scores). Additionally, the model assumes markets are mostly efficient; in reality, behavioral biases and irrational "noise traders" may distort these margins in ways that are difficult to quantify. Finally, while the use of derivatives like butterfly options is theoretically elegant, many current prediction markets lack the depth to support such complex secondary contracts. Finding simpler proxies for volatility remains a priority for applying these theories to existing platforms.

Summary updated

BibTeX
@inproceedings{freeman2021towards,
  author = {Rupert Freeman and David M. Pennock and Daniel M. Reeves and David M. Rothschild and Bo Waggoner},
  title = {{Towards a Theory of Confidence in Market-Based Predictions}},
  booktitle = {ISIPTA},
  year = {2021},
}
Figure