PersonaBench: Evaluating AI Models on Understanding Personal Information through Accessing (Synthetic) Private User Data

Juntao Tan, Liangwei Yang, Zuxin Liu, Zhiwei Liu, Rithesh Murthy, Tulika Manoj Awalgaonkar, Jianguo Zhang, Weiran Yao, Ming Zhu, Shirley Kokane, Silvio Savarese, Huan Wang, Caiming Xiong, Shelby Heinecke

TL;DR

This paper introduces PersonaBench, a novel benchmark and synthetic data generation pipeline designed to evaluate AI models' ability to understand personal user information. The authors demonstrate that current Retrieval-Augmented Generation (RAG) models struggle to effectively extract and utilize this information, even from provided documents, highlighting a significant gap in AI personalization capabilities.

What Problem Does This Paper Address?

Personalization is key for AI assistants to be truly helpful, especially when dealing with individual users' private data like conversation history or app usage. To personalize effectively, AI models need to understand a user's occupation, preferences, social connections, and more. However, the sensitive nature of this data makes it nearly impossible to obtain publicly for research. This lack of realistic, accessible data hinders the development and evaluation of AI models' personalization skills. Without standardized benchmarks, it's hard to objectively measure how well these models can actually understand and use personal information.

Methods

The core of this work is a synthetic data generation pipeline that creates realistic, yet entirely fictional, user profiles and their associated private documents. This pipeline involves several stages:

  1. Persona Sampling & Social Graph Creation: They start by sampling diverse persona descriptions and then build a social graph, connecting individuals based on plausible relationships. This ensures that social context influences individual attributes.
  2. Profile Completion: User profiles are then fleshed out with demographic, psychographic, and social information, ensuring consistency with the social graph.
  3. Private Data Synthesis: Realistic private documents are generated, including:
  4. Conversation Data: Simulated chats between connected individuals.
  5. User-AI Interaction: Logs of user interactions with an AI assistant.
  6. Purchase History: Synthesized records reflecting user preferences.
  7. Crucially, the pipeline incorporates "noise" data (irrelevant information) and simulates information updates over time to mimic real-world complexity.

The generated documents are then used to create personalized questions, for which the synthetic profiles serve as the ground truth. The AI models only have access to the documents, not the profiles themselves, simulating a real-world scenario.

Datasets

The paper introduces PersonaBench, a newly generated synthetic dataset. It comprises:

  • Synthetic User Profiles: Detailed profiles with demographic, psychographic, and social information.
  • Synthetic Private Documents: Conversations, user-AI interactions, and purchase histories.
  • Personalized Questions: A set of questions designed to test the AI's understanding of the synthetic user profiles based on the provided documents.

The authors also mention using public persona descriptions from Chan et al. (2024) to enhance the diversity of their sampled personas.

Key Insights and Ideas

  • The "Private Data" Challenge is Real: The fundamental difficulty in evaluating personalization stems from the inaccessibility of real user data. Synthetic data generation is a necessary workaround, but it needs to be highly realistic.
  • Noise is Not Just Noise: Real user data is messy. Including "noise" (irrelevant information) in the synthetic data is crucial for a realistic evaluation, as it significantly impacts model performance.
  • Current RAG is Insufficient: Even with access to all relevant documents, current RAG models struggle to answer personalized questions accurately. This suggests that simply retrieving information isn't enough; deeper understanding and inference are needed.
  • Model Size Isn't Everything: While larger retriever models generally perform better, the choice of LLM also plays a significant role, with some smaller models surprisingly outperforming larger ones in certain aspects of personalization.
  • Implicit Information is Hard: The results suggest that personal information is often conveyed implicitly, making it difficult for models to extract, even when the context is provided.

Key Results

The paper evaluates various Retrieval-Augmented Generation (RAG) pipelines. Key findings include:

  • Retrieval Performance: Even with the best retriever models (like bge-m3), recall for relevant information is around 32.5% at 50% noise, indicating a significant portion of information is missed.
  • End-to-End Performance: When evaluated on answering personalized questions, the best RAG models achieved around 50% recall when given ground-truth context. This is significantly lower than when the models had to rely on retrieved context.
  • Noise Impact: Increasing noise levels consistently degraded both retrieval and end-to-end performance, highlighting the challenge of extracting signal from noisy data.
  • LLM vs. Retriever: While better retrievers improve performance, the choice of LLM also matters. Surprisingly, GPT-40-mini sometimes outperformed GPT-4, and GPT-3.5-turbo showed comparable results to GPT-4 in some scenarios.

Limitations and Future Work

Limitations: * Restricted Release of Ground-Truth Profiles: The underlying "source of truth" profiles are not released to prevent misuse, though the generation pipeline is described in detail. * Potential for Offensive Content: Despite efforts to ensure realism, some synthetic content might be perceived as offensive or sensitive. * Inconsistencies and Unrealistic Information: Due to the automated generation process, occasional inconsistencies or unrealistic details might be present.

Future Work: The authors emphasize the need for more sophisticated methodologies and system designs to effectively extract and leverage personal information from heterogeneous and imperfect private user data. This includes developing models that can better handle noisy, fragmented, and implicitly conveyed information.

Summary updated

Abstract

Personalization is critical in AI assistants, particularly in the context of private AI models that work with individual users. A key scenario in this domain involves enabling AI models to access and interpret a user's private data (e.g., conversation history, user-AI interactions, app usage) to understand personal details such as biographical information, preferences, and social connections. However, due to the sensitive nature of such data, there are no publicly available datasets that allow us to assess an AI model's ability to understand users through direct access to personal information. To address this gap, we introduce a synthetic data generation pipeline that creates diverse, realistic user profiles and private documents simulating human activities. Leveraging this synthetic data, we present PersonaBench, a benchmark designed to evaluate AI models' performance in understanding personal information derived from simulated private user data. We evaluate Retrieval-Augmented Generation (RAG) pipelines using questions directly related to a user's personal information, supported by the relevant private documents provided to the models. Our results reveal that current retrieval-augmented AI models struggle to answer private questions by extracting personal information from user documents, highlighting the need for improved methodologies to enhance personalization capabilities in AI.
BibTeX
@inproceedings{tan2025personabench,
  author = {Juntao Tan and Liangwei Yang and Zuxin Liu and Zhiwei Liu and Rithesh Murthy and Tulika Manoj Awalgaonkar and Jianguo Zhang and Weiran Yao and Ming Zhu and Shirley Kokane and Silvio Savarese and Huan Wang and Caiming Xiong and Shelby Heinecke},
  title = {{PersonaBench: Evaluating AI Models on Understanding Personal Information through Accessing (Synthetic) Private User Data}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2025},
}
Figure