A Mechanism Design Approach to Measure Awareness

Diodato Ferraioli, Carmine Ventre, Gabor AranyiAAAI 2015 · DOI

Introduction

Determining whether a subject is "aware" of a specific stimulus remains a fundamental challenge in neuroscience and cognitive psychology. One of the most popular empirical tools for this task is Post-Decision Wagering (PDW). In a typical PDW setup, a subject performs a task (e.g., identifying a stimulus) and then wagers money on the accuracy of their response. The underlying assumption is that a high wager indicates conscious awareness, while a low wager paired with high accuracy suggests unconscious processing, such as "blindsight."

However, PDW is often criticized for its susceptibility to human behavioral noise. Factors such as risk aversion—where a subject wagers low despite being certain to avoid potential loss—or cognitive biases can lead to data that misrepresents the subject's actual state of awareness. This paper addresses these flaws by treating the psychological experiment as a problem of mechanism design, a subfield of game theory. The authors propose a formal framework to design "games" where the subject's optimal strategy is to report their true level of awareness.

The Problem: The Fragility of Standard Wagering

The authors formally demonstrate that the standard PDW protocol is only a reliable measure of awareness under extremely narrow conditions: the subject must be perfectly rational, risk-neutral, and entirely free of bias.

In reality, subjects are rarely risk-neutral. If a subject is risk-averse, they may report low confidence even when they have high awareness, simply to guarantee a small payout. Conversely, a risk-seeking subject might wager high on a guess. Because standard PDW cannot distinguish between a subject's level of awareness and their attitude toward risk, it fails as a robust scientific instrument. The paper seeks to solve this by designing mechanisms that are incentive-compatible, meaning the subject maximizes their expected utility only by telling the truth.

Theoretical Framework and Methods

The paper translates the psychological experiment into a formal mechanism design problem using several key concepts from algorithmic game theory:

  • The "Type": The subject’s internal awareness level (represented as a value between 0 and 1) is treated as their private "type."
  • The Revelation Principle: The authors focus on "direct" mechanisms where subjects are asked to report a number representing their confidence level directly.
  • Cycle-Monotonicity: This tool is used to prove whether a proposed payment rule (the wager outcome) actually incentivizes truth-telling.

By applying these tools, the researchers move beyond simple binary wagers to more complex payment functions that can account for the subject's internal state.

Key Mechanisms and the Impossibility Result

The authors introduce two primary new mechanisms to improve upon standard PDW:

  1. D-PDW (Direct PDW): This mechanism is designed to be risk-independent. Regardless of whether a subject is a cautious saver or a high-stakes gambler, their best move is to report their true awareness level. This removes the confounding variable of risk attitude from the data.
  2. S-PDW (Strict PDW): This mechanism is strictly truthful, meaning any deviation from the truth (lying) results in a strictly lower expected payout. However, the authors note a significant catch: to implement S-PDW, the experimenter must already know the subject's specific risk attitude.

Crucially, the paper establishes a fundamental Impossibility Result: it is mathematically impossible to create a mechanism that is simultaneously risk-independent and strictly truthful for measuring awareness. Researchers must choose between a mechanism that works for everyone regardless of risk (D-PDW) or one that provides a stronger incentive for truth but requires prior knowledge of the subject's psychology (S-PDW).

Addressing Bias and Quantifying Confidence

Beyond risk, the authors address the issue of bias—where subjects expect certain outcomes based on previous trials rather than current perception. They propose a "two-step" filtering approach: by requiring subjects to report their expectations before a trial begins, researchers can identify and filter out trials where the subject's response is likely driven by pattern-matching rather than actual awareness.

Furthermore, the paper provides a statistical foundation for these experiments using the Hoeffding bound. This allows researchers to calculate exactly how many trials are required to map a subject's "awareness distribution" with a specific level of mathematical confidence.

Conclusion and Future Directions

While this paper provides a robust mathematical foundation for measuring consciousness, it relies on the assumption of rationality. The mechanisms assume that subjects are utility maximizers who can calculate the best strategy to maximize their earnings. In practice, human subjects may be irrational or may struggle to understand complex payment rules.

The authors suggest that future work should focus on translating these theoretical "direct" mechanisms into "indirect" versions—simpler experimental tasks that are easier for humans to navigate while retaining the same mathematical guarantees of truthfulness. By bridging the gap between game theory and cognitive science, this approach offers a more rigorous path toward quantifying the elusive nature of human awareness.

Summary updated

BibTeX
@inproceedings{ferraioli2015mechanism,
  author = {Diodato Ferraioli and Carmine Ventre and Gabor Aranyi},
  title = {{A Mechanism Design Approach to Measure Awareness}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2015},
  doi = {10.1609/aaai.v29i1.9298},
}
Figure