LifeVal: Evaluating AI Model Performance on Personalized Agentic Consumer Tasks

Jenny Yang, Elizabeth Proehl, Anjan Bharadwaj, Emily Fan, Charlotte Cole, James Kim, Noam Brown, Phoebe Thacker, Amelia Glaese, Tejal PatwardhanarXiv 2026 · arXiv

TL;DR

LifeVal is a new benchmark from OpenAI designed to evaluate AI agents on complex, highly personalized consumer tasks—ranging from managing chronic health care to booking travel with credit card points. By grounding the benchmark in a $1.7 trillion "willingness-to-pay" economic framework, the authors ensure that AI performance is measured against the real-world tasks that humans value most.

What Problem Does This Paper Address?

While AI models are increasingly capable, existing benchmarks often fall into two traps: they are either too academic (single-turn Q&A) or too generic (tasks that don't require personal context). To build a true "digital superassistant," we need to measure how well an agent can handle "agentic" workflows—tasks that require using a computer, browsing the web, and accessing a user's specific history and preferences.

The authors argue that "time spent" on a task is a poor proxy for its value. For instance, humans spend a lot of time sleeping or eating, but they wouldn't pay an AI to do those things. Instead, LifeVal focuses on the "administrative burden" of life: high-stakes, infrequent, or complex tasks like navigating insurance, planning a move, or finding specialized childcare.

Methods

The researchers took a multi-step approach to build a benchmark that is both economically relevant and technically rigorous:

  1. Economic Prioritization: They conducted a survey of over 3,800 consumers to determine their "willingness-to-pay" (WTP) for AI assistance across 148 categories, cross-referencing this with global time-use studies.
  2. Expert Task Creation: They recruited "expert" task writers who had personally managed these specific tasks in their own lives for over a year. These writers created tasks based on their own real (but de-identified) ChatGPT histories, memories, and files.
  3. Agentic Environment: Each task is paired with a Reinforcement Learning Environment (RLE)—a synthetic, controlled space where the model can actually take actions like clicking buttons, filling out forms, and browsing a cached version of the web.
  4. Rigorous Quality Control: Tasks underwent three rounds of human review (Peer, Lead, and Final Sign-off) plus a "Model QC" phase where an AI attempted the task to ensure it was actually solvable with the provided context.
Figure 3: Task structure

Datasets

  • LifeVal: The primary contribution, consisting of 169 complex, personalized tasks.
  • ATUS & MTUS: The American Time Use Survey and Multinational Time Use Study were used to establish a baseline of how humans spend their time.
  • WTP Survey: A new dataset of consumer valuations for AI assistance, totaling an estimated $1.7 trillion in annual market value for the US alone.

Key Insights and Ideas

  • The Personalization Gap: The study found that "Additional Task Context" (specific details provided for the task) is far more useful to current models than "Past Conversations." Models still struggle to retrieve and apply relevant information buried deep in long chat histories.
  • The Alignment Paradox: As models become more "aligned" (like GPT-6 Astra), they actually fail more often by reporting an "inability to find information" or leaving actions uncompleted. This suggests that stricter safety or accuracy boundaries might make agents more hesitant to take the "leap" required for complex agentic tasks.
  • Willingness-to-Pay as a Metric: By using WTP, the benchmark prioritizes "high-consequence" tasks (like financial management) over "high-frequency" tasks (like social media scrolling), which better reflects the utility of a professional-grade assistant.
Figure 1: Example LifeVal Tasks. Any personal information in the paper, including any images with personal information, has been deidentified or synthetically generated to protect our task creators’ privacy.

Key Results

The researchers evaluated a lineage of models from GPT-5.2 to GPT-6 Astra.

  • Performance Gains: There is a clear upward trend in reward/success rates as models evolve, particularly in their ability to handle tool calls and long-token rollouts.
  • Failure Modes: GPT-5.2 was the only model to frequently "run out of tokens" before finishing. In contrast, the most common failure mode for newer models is "Incorrect information or recommendation," often because the model missed a key detail in the user's personal history.
  • Grading Accuracy: The automated grading system (using GPT-5.6) showed a high correlation ($r = 0.864$) with human expert graders, suggesting that automated evaluation of these subjective, personalized tasks is becoming viable.
A breakdown of failure modes across different model versions.

Limitations and Future Work

The authors acknowledge several limitations:

  • Demographics: The WTP survey was US-centered and conducted in English, potentially overlooking tasks valued in other cultures or languages.
  • Static Environments: While RLEs provide reproducibility, they use cached web data. Real-world agents will face a "live" web that is constantly changing.
  • Self-Reporting Bias: WTP is a proxy for value based on what people say they would pay, which may differ from their actual behavior once these tools are widely available.

Future work will likely focus on improving "long-term memory" retrieval, allowing agents to better utilize years of conversation history rather than just the immediate context.

Summary updated

Abstract

We introduce LifeVal, a benchmark evaluating AI model performance on personalized agentic consumer tasks. LifeVal covers the most valuable life tasks that a personalized digital superassistant could automate, representing over 1.7 trillion US dollars in annual willingness-to-pay across 148 consumer task categories (e.g., managing finances, moving apartments, searching for a job, finding childcare). To construct our activity distribution, we conducted a new survey of willingness-to-pay for a digital superassistant across a demographically representative sample of over 3,800 consumers, cross-referencing our results against the American Time Use Survey (Bureau of Labor Statistics, 2025) and the Multinational Time Use Study (Fisher et al., 2025). For the most valuable consumer activities, we recruited task creators to create AI evaluation tasks personalized using real ChatGPT conversations and memories, reference files, and personal context. Task creators had to pass our expertise threshold (including personally completing the task multiple times and seeing the results through in their real lives for over a year) and also write a rubric, ensuring that our evaluation measured the best personalized completion for that task. All tasks had to pass at least three rounds of human quality-control before acceptance into our final evaluation sets.
BibTeX
@misc{yang2026lifeval,
  author = {Jenny Yang and Elizabeth Proehl and Anjan Bharadwaj and Emily Fan and Charlotte Cole and James Kim and Noam Brown and Phoebe Thacker and Amelia Glaese and Tejal Patwardhan},
  title = {{LifeVal: Evaluating AI Model Performance on Personalized Agentic Consumer Tasks}},
  year = {2026},
  eprint = {2611.06687},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.06687},
}
Figure