Testing Under Strategic Manipulation: Mechanism Design for Human and AI Institutions

Xiaoyun Qiu, Liren Shan

TL;DR

This paper explores how to design testing systems—like job screenings or financial audits—when candidates can strategically "game" the results. The authors prove that a fixed-order sequence of "stringent" tests is often superior to randomized ones, and they demonstrate that AI-driven personalization (using upfront communication to tailor the testing path) can achieve perfect selection efficiency.

What Problem Does This Paper Address?

In a perfect world, a test reveals an agent's true "qualified" status. In reality, agents are strategic: they can manipulate their attributes at a cost to pass. Think of a bank "window dressing" its balance sheet right before a quarterly audit to meet liquidity ratios, or a job applicant optimizing their resume keywords for an ATS without actually possessing the skills.

The challenge for "principals" (the institutions doing the testing) is that they often have multiple criteria to check. The authors ask: How should these tests be organized? Should they be administered by independent agencies or a joint bureaucracy? Should the order be fixed or randomized? And can modern AI tools, which allow for interactive "cheap-talk" before a test, help us filter candidates more effectively?

Methods

The authors use a mechanism design framework, modeling agents and tests in a multi-dimensional attribute space (specifically $\mathbb{R}^d$).

  • Attributes & Costs: Agents have a "true type" (their actual attributes) but can move to a "manipulated type" at a cost defined by the Euclidean distance between the two.
  • Institutional Models: They compare a Joint Bureaucracy (which coordinates test order and disclosure) against Independent Bureaucracies (where agents might choose the order themselves).
  • Geometric Analysis: Much of the proof relies on a "two-step manipulation" insight. An agent might pass Test A, then shift their attributes again to pass Test B. The authors use geometric reflections to visualize the "cheapest" path for a strategic agent to pass both tests.
  • Personalization: They model a "menu of procedures" where an agent sends a costless message (cheap-talk) to an AI, which then assigns them to a specific testing sequence.

Datasets

As a theoretical paper in mechanism design, no empirical datasets were used. The results are derived through mathematical proofs and geometric constructions applicable to any distribution of agent types ($F$).

Key Insights and Ideas

  • The Power of Fixed Order: Contrary to the intuition that randomness prevents gaming, the paper finds that fixed-order sequential tests are optimal. Why? Because randomness makes the "barrier to entry" unpredictable, forcing the principal to lower test stringency to avoid failing qualified candidates. A fixed order allows the principal to set much more aggressive (stringent) thresholds.
  • Stringency as a Filter: The optimal tests are often "stricter" than the actual requirement. By raising the bar, the principal ensures that only those who are truly qualified (or those for whom manipulation is prohibitively expensive) can pass.
  • AI as a Triage Tool: The authors show that "personalization" via upfront communication (e.g., a chatbot interaction) can achieve the first-best outcome—selecting all qualified candidates and zero unqualified ones. By offering a "menu" of different testing paths, the principal can cover the entire "qualified region" of the attribute space without letting manipulators slip through the cracks.
  • Manipulation vs. Investment: The paper makes a crucial distinction between "gaming" (temporary, superficial changes) and "investment" (changes that become the agent's new true attributes). In an investment setting, randomized tests actually become superior because the uncertainty encourages agents to improve across all dimensions simultaneously.

Key Results

  • Theorem 1 (Optimality of Fixed Order): For any distribution of candidates, the optimal joint bureaucracy uses a fixed-order procedure with stringent tests.
  • Theorem 2 (The Value of Personalization): With upfront communication (cheap-talk), there exists a fixed-order mechanism that achieves 100% selection efficiency.
  • Proposition 1 (Control Matters): Controlling the test order allows a joint bureaucracy to use a wider array of tests (including non-parallel tests) that independent agencies cannot safely use without being gamed.
  • Theorem 3 (Investment Setting): If the goal is to encourage genuine improvement (investment) and the criteria are sufficiently distinct (angle $> 30^\circ$), a random-order mechanism without disclosure is the "first-best" solution.

Limitations and Future Work

  • Linearity Assumptions: The model assumes tests are linear classifiers (hyperplanes) and manipulation costs are Euclidean. Real-world manipulation might have non-linear costs or categorical constraints.
  • Binary Outcomes: The paper focuses on pass/fail outcomes. Future work could explore continuous scoring systems.
  • Cost of Testing: While the paper mentions that personalization can reduce the "testing workload," it doesn't deeply model the operational costs of running complex, multi-stage AI-mediated bureaucracies.
  • Dynamic Environments: The model is static. In reality, agents learn over time, and the "cost" of manipulation might decrease as "gaming" techniques become standardized.

Summary updated

Abstract

We study how the design of testing institutions, encompassing both the tests themselves and the procedures used to administer them, shapes selection outcomes in environments with multiple criteria and strategic agents. We model the testing agency as either a set of independent bureaucracies (each test administered separately) or a joint bureaucracy (where test order and personalization can be coordinated). Our mechanism design analysis shows that under a joint bureaucracy, fixed-order sequential mechanisms with stringent tests are optimal for maximizing the probability mass of qualified candidates selected. Furthermore, we demonstrate that personalizing tests through upfront communication, now increasingly feasible via AI and automation, can select all qualified candidates. Finally, we compare institutional settings and quantify the value of controlling test order, showing that the benefit depends critically on the distribution of testees and the stringency of optimal tests. Our results contribute to the design of robust, efficient, and fair testing systems in both human and AI-mediated environments.
BibTeX
@inproceedings{qiu2026testing,
  author = {Xiaoyun Qiu and Liren Shan},
  title = {{Testing Under Strategic Manipulation: Mechanism Design for Human and AI Institutions}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2026},
  doi = {10.1609/aaai.v40i20.38772},
}
Figure