No article text was recorded for this version.
human evaluation scalability
Version 1 of 179 papers79 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Evaluating the Performance of Large Language Models via Debates
- Blog: QRFA: A Data-Driven Model of Information-Seeking Dialogues
- Blog: Tuning LLM Judge Design Decisions for 1/1000 of the Cost
- Blog: AVA: an Automatic eValuation Approach for Question Answering Systems
- Blog: AutoEval-ToD: Automated Evaluation of Task-oriented Dialog Systems
- Blog: Can Large Language Models Be an Alternative to Human Evaluations?
- Blog: PONE: A Novel Automatic Evaluation Metric for Open-domain Generative Dialogue Systems
- Blog: On the Evaluation of Capability Estimation Methods for Large Language Models
- Blog: Language Model Transformers as Evaluators for Open-domain Dialogues
- Blog: Investigating Evaluation of Open-Domain Dialogue Systems With Human Generated Multiple References
- Blog: An In-depth Investigation of User Response Simulation for Conversational Search
- Blog: Evaluating LLM-based Agents for Multi-Turn Conversations: A Survey
- Blog: Assessing and Verifying Task Utility in LLM-Powered Applications
- Blog: Quantifying Zero-shot Coordination Capability with Behavior Preferring Partners
- Blog: An Evaluation of Chinese Human-Computer Dialogue Technology
- Blog: Evaluating Mixed-initiative Conversational Search Systems via User Simulation
- Blog: Towards Automatic Evaluation for LLMs' Clinical Capabilities: Metric, Data, and Algorithm
- Blog: SimulatorArena: Are User Simulators Reliable Proxies for Multi-Turn Evaluation of AI Assistants?
- Blog: Large Language Models as Automated Aligners for benchmarking Vision-Language Models
- Blog: Evaluating Conversational Agents with Persona-driven User Simulations based on Large Language Models: A Sales Bot Case Study
- Blog: Judging the Judges: A Systematic Investigation of Position Bias in Pairwise Comparative Assessments by LLMs
- Blog: LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks
- Blog: MMBench-Live: A Continuously Evolving Benchmark for Multimodal Models
- Blog: What Limits Virtual Agent Application? OmniBench: A Scalable Multi-Dimensional Benchmark for Essential Virtual Agent Capabilities
- Blog: An Evaluation Framework for Conversational Information Retrieval Using User Simulation
- Blog: HELP ME THINK: A Simple Prompting Strategy for Non-experts to Create Customized Content with Models
- Blog: Spoken Dialog Challenge 2010: Comparison of Live and Control Test Results
- Blog: Spot The Bot: A Robust and Efficient Framework for the Evaluation of Conversational Dialogue Systems
- Blog: ESC-Eval: Evaluating Emotion Support Conversations in Large Language Models
- Blog: Understanding the Effectiveness of Very Large Language Models on Dialog Evaluation
- Blog: Investigating the Impact of Pre-trained Language Models on Dialog Evaluation
- Blog: Deconstruct to Reconstruct a Configurable Evaluation Metric for Open-Domain Dialogue Systems
- Blog: ChatMatch: Evaluating Chatbots by Autonomous Chat Tournaments
- Blog: Unblocking Fine-Grained Evaluation of Detailed Captions: An Explaining AutoRater and Critic-and-Revise Pipeline
- Blog: Survey on evaluation methods for dialogue systems
- Blog: IQA-EVAL: Automatic Evaluation of Human-Model Interactive Question Answering
- Blog: Joint Goal Segmentation and Goal Success Prediction on Multi-Domain Conversations
- Blog: Evaluating Conversational Recommender Systems via User Simulation
- Blog: SelfPrompt: Autonomously Evaluating LLM Robustness via Domain-Constrained Knowledge Guidelines and Refined Adversarial Prompts
- Blog: A Hierarchical Domain Model-Based Multi-Domain Selection Framework for Multi-Domain Dialog Systems
- Blog: Towards Automatic Evaluation of Dialog Systems: A Model-Free Off-Policy Evaluation Approach
- Blog: BotChat: Evaluating LLMs' Capabilities of Having Multi-Turn Dialogues
- Blog: MEEP: Is this Engaging? Prompting Large Language Models for Dialogue Evaluation in Multilingual Settings
- Blog: Reduce Human Labor On Evaluating Conversational Information Retrieval System: A Human-Machine Collaboration Approach
- Blog: ConDABench: Interactive Evaluation of Language Models for Data Analysis
- Blog: PoE: a Panel of Experts for Generalized Automatic Dialogue Assessment
- Blog: Learning Evaluation Models From Large Language Models for Sequence Generation
- Blog: A review of evaluation techniques for social dialogue systems
- Blog: X-TURING: Towards an Enhanced and Efficient Turing Test for Long-Term Dialogue Agents
- Blog: Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systems
- Blog: Define, Evaluate, and Improve Task-Oriented Cognitive Capabilities for Instruction Generation Models
- Blog: Direct Judgement Preference Optimization
- Blog: Towards Holistic and Automatic Evaluation of Open-Domain Dialogue Generation
- Blog: ConvCodeWorld: Benchmarking Conversational Code Generation in Reproducible Feedback Environments
- Blog: Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systems
- Blog: Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluation
- Blog: Scaling Up Active Testing to Large Language Models
- Blog: How BERT Speaks Shakespearean English? Evaluating Historical Bias in Masked Language Models (short paper)
- Blog: Conversational Search: From Fundamentals to Frontiers in the LLM Era
- Blog: Predicting User Engagement Status for Online Evaluation of Intelligent Assistants
Showing 60 of 79 papers.
History & citation
Current · Permanent link
Cite this direction
Latest version (a living page):
@misc{lacuna_human_evaluation_scalability_778e8bdd,
title = {human evaluation scalability},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/human-evaluation-scalability/rdr_778e8bdddc6649bca2b5b1eaaa51a713},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}This exact version (version 1, frozen text):
@misc{lacuna_human_evaluation_scalability_778e8bdd_v1,
title = {human evaluation scalability},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/human-evaluation-scalability/rdr_778e8bdddc6649bca2b5b1eaaa51a713/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}