Evaluating the Joint Performance of Human-AI Teams

98 concepts71 papers2 proposals

Imagine an AI assistant that achieves a perfect score on a static data science benchmark. When deployed in a real-world corporate environment, however, the system fails completely. Why does this happen? Traditional benchmarks evaluate models in a vacuum, assuming a single-turn, fully autonomous execution. In reality, professional tasks are iterative, ambiguous, and deeply collaborative.

As artificial intelligence transitions from isolated tools to interactive partners, a major paradigm shift is underway: researchers are moving away from model-centric metrics toward interaction-centric evaluation frameworks that measure the joint performance of the human-AI team.

The Blind Spots of Isolated Metrics

Traditional machine learning evaluation prioritizes internal performance factors like computational speed, accuracy, and F1-scores. However, as Siddharth Srivastava et al. point out, these metrics fail to account for how effectively or safely a system can be operated by diverse human users in real-world settings. Standardized leaderboards often reward marginal gains in algorithmic performance while completely ignoring the broader system context, such as user interfaces and social workflows (Matthew Lease et al.).

This disconnect is particularly glaring in creative and exploratory tasks. Standard natural language processing benchmarks score models on their ability to generate text in isolation, overlooking the interactive process of how humans and machines co-create content (Percy Liang et al.). Relying on static, single-turn scoring fails to capture the iterative nature of real-world workflows, where a user might reject five bad ideas to find one inspired suggestion (Judy Hanwen Shen et al.).

Shifting Focus to Joint Outcomes and Performativity

When a human holds the ultimate decision-making authority, evaluating the model's intermediate precision becomes less meaningful than measuring its impact on final outcomes. Sriraam Natarajan et al. argue for a shift to outcome-based metrics—such as patient health or investment success—prioritizing the quality of the end result over the intermediate accuracy of the model's suggestions.

[Isolated Evaluation]   Model Prediction ─────────────────────────────► Accuracy Score

[Joint Evaluation]      Model Suggestion ──► Human Decision ──► Outcome ──► System Success

Furthermore, deployed models operate in performative environments: their predictions actively alter human behavior, creating feedback loops that bias subsequent data. To address this, Jean Feng et al. propose causal frameworks that evaluate the joint performance of the algorithm and the human decision-maker together, rather than treating standalone model accuracy as an independent variable.

A Tripartite Framework for Interaction-Centric Evaluation

To build a complete picture of human-AI synergy, the authors of Embedding artificial intelligence in society: looking beyond the EU AI master plan using the culture cycle propose a tripartite evaluation strategy. This framework balances technical quality metrics, co-action effectiveness, and subjective human experience.

1. Co-Action and Communication Effectiveness

In a true partnership, an AI must manage ambiguity and integrate ongoing human feedback. Irene Testini et al. emphasize the need for interactive benchmarking environments that measure an agent's ability to request clarification and adapt to iterative human course-corrections. Without metrics that capture helpfulness and context maintenance across multi-step instructions, an agent cannot be reliably vetted for real-world collaboration (Julia Kiseleva et al.).

2. Cognitive Friction and Flow

High-performing models can still cause system failures if they disrupt human cognitive processes. For instance, Lubaina Arsiwala-Scheppach et al. utilize eye-tracking technology to demonstrate that poorly integrated AI decision support systems can fragment an expert's optimized visual strategy, increasing task time and cognitive friction.

Evaluating the joint system requires quantifying these psychological and physiological impacts, such as technostress and cognitive overload (Magdalena Wekenborg et al.). Conversely, successful collaborative assistants must be evaluated on their ability to preserve a user's "flow state" (Tamara Babaian et al.) and maximize the rate of insight generation rather than just raw query execution speed (Philipp Eichmann et al.).

3. Subjective Trust and Long-Term Impact

Human users typically evaluate social intent and warmth before they ever assess competence; a lack of perceived warmth can prevent the establishment of trust even if the agent is highly capable (Aniket Bera et al.). Effective evaluation must therefore integrate user-centric dimensions like trust calibration, perceived reliability, and long-term satisfaction (Volkmar Pipek et al.; Kaike Zhang et al.).

As Diyi Yang argues in Human-AI Interaction in the Age of Large Language Models, the transition to collaborative AI necessitates measuring how these interactions reshape human capabilities and behaviors over time, rather than treating user capability as a static baseline.

Methodological Challenges

Building robust, interaction-centric benchmarks introduces unique experimental hurdles:

  • Reproducibility and Cost: Evaluating systems through direct interaction with human subjects is logistically difficult, expensive, and nearly impossible to replicate consistently across different laboratories (Tin Dizdarevic et al.). This drives the need for high-fidelity "human-like" proxy agents to simulate interactive environments.
  • Credit Assignment: In an interactive system, a user's own adaptive creativity can frequently compensate for a system's flaws or lack of originality. Julian Togelius et al. note that isolating the system's distinct creative contribution from the user's compensatory interaction remains an open methodological challenge.
  • High Stakes Environments: In domains like disaster recovery or medicine, poorly calibrated trust can have catastrophic consequences. Metrics must explicitly measure how well an AI framework supports human decision-making under pressure, ensuring that the system calibrates user trust accurately rather than encouraging blind reliance (Pulkit Verma et al.).

Key Foundational Work

Writer's Aid: Using a Planner in a Collaborative Interface (2003) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_human_centric_performance_metrics_d3a46c90,
  title = {human-centric performance metrics},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/human-centric-performance-metrics/rdr_d3a46c906f3f4d48beb9c35e25bb25b5},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_human_centric_performance_metrics_d3a46c90_v1,
  title = {human-centric performance metrics},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/human-centric-performance-metrics/rdr_d3a46c906f3f4d48beb9c35e25bb25b5/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake