TL;DR
This paper introduces "Know Thyself," a novel framework for evaluating LLM-based persona agents' self-awareness and domain knowledge. It addresses the critical need for robust validation of these agents, which are increasingly used as cost-effective proxies for human users in various applications. The study benchmarks three LLMs across three domains, revealing that while current models show promise, there's still room for improvement in consistency and diversity of responses.
What Problem Does This Paper Address?
Large Language Models (LLMs) are becoming adept at simulating human behavior, personality, and language, leading to the development of "persona agents." These agents are valuable for tasks like data annotation, surveys, and A/B testing, acting as digital stand-ins for real users. However, as these agents are customized for specific roles and domains, it's crucial to ensure they possess accurate "knowledge awareness" – understanding their own persona and the relevant domain knowledge. Currently, there's a lack of standardized methods to comprehensively evaluate this awareness, particularly concerning consistency, relevance, and diversity. Existing evaluations often focus on individual persona traits rather than the agent's ability to utilize that persona information effectively, especially in domain-specific contexts. This gap hinders the development of trustworthy and robust AI applications that rely on these synthetic personas.
Methods
The authors propose a generic evaluation framework called "Know Thyself," designed to be adaptable to any persona and domain. The approach involves:
- Persona Agent Definition: Domain experts define target user profiles with specific attributes.
- Question Generation: LLMs are used to generate a diverse set of questions covering two dimensions:
- Self-World: Fact-based questions about persona attributes (name, age, nationality, etc.) and open-ended questions probing common sense and inferencing related to the persona.
- Domain: Fact-based questions testing knowledge of a specific domain and open-ended questions requiring domain-specific reasoning.
- Response Generation: LLM agents, assigned specific personas via system prompts, generate responses to the curated questions.
- Evaluation: Responses are assessed using a rubric by both human experts and an LLM evaluator (GPT-4). Metrics include correctness, relevance to profile, logical integrity, and relevance to domain.
The study also experiments with advanced prompting strategies like Basic, Chain-of-Thought (CoT), ReAct, and Reflexion to assess their impact on performance.
Datasets
The paper does not introduce a new benchmark dataset in the traditional sense. Instead, it describes the generation of evaluation questions and personas for their experiments.
- Personas: 25 sample personas were created across three domains: Air Travel Agency, Gym Gear Manufacturer, and Video Game Production.
- Questions: A total of 104 questions for self-world evaluation and 55 for domain evaluation were generated. These questions were a mix of fact-based and semi-fact-based (open-ended) to assess memorization, inferencing, and diversity.
Key Insights and Ideas
- The "Self-World" is Crucial for Trust: The authors posit that an agent's awareness of its own persona and the world it inhabits is fundamental to building user trust.
- Diversity is Elusive: Despite using advanced prompting strategies, achieving diverse responses from LLM persona agents remains a significant challenge. This is a critical limitation for applications like surveys where varied perspectives are needed.
- Smaller Models Can Be More Divergent: Surprisingly, smaller models like Gemma exhibited more diverse responses than larger models like GPT-40, potentially due to less stringent guardrails.
- LLM Evaluators Show Promise: The study found a high correlation between evaluations performed by human experts and an LLM evaluator (GPT-4), suggesting LLMs can be effective tools for large-scale evaluation.
- Prompting Matters, But Not Universally: While advanced prompting strategies can improve accuracy, no single strategy consistently outperformed others across all models and domains. Basic prompting sometimes yielded comparable or even better results for certain models.
Key Results
- Accuracy Varies: GPT-40 achieved the highest accuracy (average 78.6%) in self-world and domain knowledge assessment, while Gemma had the lowest (average 59.37%).
- Domain Knowledge is Generally Strong: Persona agents demonstrated sufficient domain background knowledge, with performance differences across models and prompting strategies being less significant in domain-specific evaluations.
- Prompting Impact: Providing multiple-choice options during prompting generally led to better accuracy than direct answer prompts.
- Mitigation Strategies Help: Simple prompt-based mitigation strategies like Multiple Choice Questions (MCQ) and Yes/No Questions (YN) showed significant improvements in accuracy compared to basic prompting. For instance, accuracy for Mixtral improved from 39.14% to 52.69% with MCQ, and for GPT-40 from 82.02% to 91.96% with Reflexion.
Limitations and Future Work
Limitations:
- Limited Scope: The study used a subset of LLMs, prompting strategies, and persona descriptions due to infrastructure and budget constraints. This may not fully represent the diversity of the LLM landscape or real-world populations.
Future Work:
- Enhancing Diversity: Developing strategies to improve divergent thinking and response diversity in persona agents is a key future direction.
- Mitigation Strategies: Further investigation into prompt-based mitigation strategies for improving persona agents' performance, especially in higher-order thinking skills.
- Model Diversity and Trade-offs: Analyzing model diversity, exploring ways to improve it, and studying the trade-off between diversity and performance.
- Broader Evaluation: Expanding the study to include a more comprehensive set of LLMs, domains, and personas.
Summary updated