When Does Design Thinking Work? A Convergence Theorem for the Iterate-Test Core

Paweł Kuraś, Adrian Michalski, Alicja Gerka, Patryk OrganiściakarXiv 2026 · arXiv

TL;DR

This paper provides the first formal mathematical proof for why the "iterate–test" core of design thinking works. By modeling design as a search over a landscape of possible features, the authors demonstrate that while the process eventually converges to a local optimum, real-world success depends on balancing the creation of new prototypes with the rigorous re-testing of existing favorites to overcome the "optimizer’s curse."

What Problem Does This Paper Address?

Design thinking (DT) is a staple of modern innovation, taught everywhere from Stanford’s d.school to corporate boardrooms. However, despite its popularity, DT has long lacked a rigorous theoretical foundation. Critics often describe it as "thinly grounded," relying on small-scale qualitative studies rather than a formal theory of why it should work.

The authors tackle a specific, narrow question: Under what exact conditions does the iterative loop of ideation, prototyping, and testing provably improve what a team builds? By formalizing this "inner loop," they aim to provide product managers with clear, evidence-based guidelines on how to budget and manage iterative testing.

Methods

The researchers model design thinking as a two-level process:

  1. The Outer Level: Choosing a "frame" (defining the problem and what counts as a solution).
  2. The Inner Loop: A repeated cycle of ideation, prototyping, and testing within that fixed frame.
Figure 1: The two levels of the model. Empathising and defining produce a frame, a set of features F and a utility U. Within the frame the team repeats a round of ideation, prototyping and testing, and the theorem concerns this loop. A reframe (dashed) returns the team to the outer level and starts a new framing epoch.

The core of the paper is Theorem 1, which proves that the prototype a team believes to be best will converge "almost surely" to a local optimum under five specific conditions: a finite frame, local ideation, persistent discovery of improvements, unbiased repeated testing, and the retention of tested prototypes.

To bridge the gap between asymptotic theory and practical reality, they used the notation defined in the paper to run finite-horizon simulations.

Table 1: Notation used to formalize the design thinking process.

Datasets

Because real-world design data is often proprietary or inconsistent, the authors used three classes of synthetic "landscapes" to evaluate their model:

  • DVF Landscapes: Based on the classic Desirability, Viability, and Feasibility criteria.
  • NK Rugged Landscapes: A standard model in evolutionary biology and organizational search where feature interactions create complex, multi-peaked utility surfaces.
  • Clustered Modular Architectures: Landscapes that simulate products with distinct, interacting components.

These simulations allowed the authors to know the "true" utility of every possible prototype, enabling them to measure exactly how much value a team "leaves on the table" during a project.

Key Insights and Ideas

  • The Optimizer’s Curse: Teams systematically overestimate the value of their favorite concept by 4% to 8%. This happens because the "favorite" is often chosen because it received a "lucky" high score in a noisy test, rather than being inherently superior.
  • The "Test Everything Once" Trap: The simulations revealed that showing many different prototypes to users only once is the worst possible use of a development budget. Without re-testing, teams cannot distinguish a truly great idea from a mediocre one that just had a good day in the lab.
  • Pre-screening Over Sophistication: Investing in rough, low-fidelity pre-screening of ideas before building physical prototypes delivers greater performance gains than using sophisticated testing algorithms later in the process.
  • The Limits of Iteration: Iterative testing is excellent at climbing a hill, but it won't tell you if you're on the wrong mountain. The loop itself does not signal when a fundamental "reframe" of the problem is needed.

Key Results

  • Optimal Allocation: For a budget of five tests, the most effective strategy is to build one to three new concepts and use the remaining tests to re-verify leading concepts.
  • Bias Threshold: Interestingly, confirmation bias (favoring a current design) does little harm as long as the bias is smaller than the measurement noise of the tests.
  • Convergence vs. Speed: While the theorem guarantees convergence to a local optimum in the limit, these limits are rarely reached in practice. Success is determined more by the "climb rate"—how fast a team can identify and refine promising variants.

Limitations and Future Work

The model is normative, meaning it describes how a perfectly consistent team should behave, which real-world teams only approximate. It also treats test results as simple numerical utilities, ignoring the qualitative "reasons" (e.g., why a user disliked a feature) that often guide human designers.

Future research could explore correlated beliefs—where testing one prototype teaches the team about similar variants—which would likely speed up the recognition of local optima. Additionally, the authors suggest that field data is needed to see if real-world teams actually satisfy the five conditions required for the convergence guarantee to hold.

Summary updated

Abstract

Design thinking is widely taught as an iterative loop of ideation, prototyping, and testing, yet conditions under which such a loop must improve innovation outcomes remain unspecified. Taking problem framing as given, we formalize the iterate-test core as search over a finite prototype space whose utility is learned through noisy evaluations. We prove that the prototype a team believes to be best converges almost surely to a local optimum under five conditions: finite framing, local ideation, non-summable discovery rates, unbiased repeated testing, and retention of tested prototypes. Bounded test bias or criteria drift degrades the guarantee by a bounded amount, and multiple stakeholders yield convergence to a locally Pareto-optimal set. Finite-horizon simulations across three landscape classes (additive valuation, NK rugged landscapes, and clustered modular architectures) show that asymptotic limits are rarely reached in practice. Teams face an optimizer's curse, overestimating their chosen concept by 4-8% and frequently discarding superior prototypes. Testing every prototype once proves to be the worst allocation; optimal performance requires building one to three prototypes per five tests and repeatedly re-testing leaders. Pre-screening concepts yields greater performance gains than tuning testing policies.
BibTeX
@misc{kuras2026when,
  author = {Paweł Kuraś and Adrian Michalski and Alicja Gerka and Patryk Organiściak},
  title = {{When Does Design Thinking Work? A Convergence Theorem for the Iterate-Test Core}},
  year = {2026},
  eprint = {2611.03034},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.03034},
}
Figure