When evaluating natural language generation (NLG) systems, automated metrics like BLEU, ROUGE, and BERTScore are the default choice. They are cheap, deterministic, and scale infinitely. However, as language models move from simple pattern matching to complex reasoning, open-ended dialogue, and high-stakes domain generation, a critical gap has widened: automated metrics frequently fail to capture human utility, accuracy, and nuance.
The core issue is that automated metrics typically measure statistical overlap or vector similarity against a reference text, rather than evaluating whether the generated text actually achieves its communicative goal. This systemic reliance on automated proxies creates a blind spot across the entire spectrum of modern NLP research.
The Surface-Level Alignment Problem
In open-ended tasks like dialogue and summarization, a model can generate a response that is completely fluent and contextually appropriate but shares zero words with a ground-truth reference. Conversely, a model can generate a repetitive or nonsensical response that scores highly on n-gram overlap.
For example, in dialogue generation, Junya Takayama et al. demonstrate in Relevant and Informative Response Generation using Pointwise Mutual Information that while metrics like BLEU and NIST show statistical improvements, they cannot guarantee that responses are genuinely informative or content-rich for a human interlocutor. This sentiment is echoed by Ana Valeria González-Garduño in Reinforcement Learning for Improved Low Resource Dialogue Generation, who notes that subjective assessments are ultimately mandatory to validate the quality and meaning of low-resource dialogue systems. Even when models achieve state-of-the-art results on standard benchmarks—such as the neural surface realization pipeline proposed by Henry Elder et al. in Designing a Symbolic Intermediate Representation for Neural Surface Realization—the authors emphasize that human evaluation remains irreplaceable for identifying specific structural failure modes.
This pattern repeats across diverse tasks:
- Summarization and Simplification: In spoken document summarization, Potsawee Manakul et al. rely on ROUGE scores in Abstractive Spoken Document Summarization Using Hierarchical Model with Multi-Stage Attention Diversity Optimization but acknowledge that human judgment is required to truly gauge coherence and fluency. Similarly, Kaijie Mo et al. point out in ExpertEase: A Multi-Agent Framework for Grade-Specific Document Simplification with Large Language Models that automated readability tools and agent-based feedback cannot fully capture the nuanced comprehension of a target human audience.
- Cross-Lingual Transfer and Labeling: Mehrad Moradshahi et al. observe in Zero and Few-Shot Localization of Task-Oriented Dialogue Agents with a Distilled Representation that metrics like BLEU and SER fall short of assessing grammatical correctness in cross-lingual settings. In intent discovery, Grant Anderson et al. note in Automated Human-Readable Label Generation in Open Intent Discovery that automated metrics cannot definitively measure whether a generated label is practically readable and useful for end-users.
- Multimodal and Simulation Tasks: In image captioning, Marimuthu Kalimuthu et al. argue in Fusion Models for Improved Image Captioning that human evaluation is necessary to draw definitive conclusions about caption quality. When training language models via user simulators, Kaustubh Dhole highlights in KAUCUS - Knowledgeable User Simulators for Training Large Language Models that automated metrics fail to capture the complex nuances of human interaction and preference.
High-Stakes Domains and the Safety Gap
The limitations of automated metrics shift from inconvenient to dangerous when applied to high-stakes domains like medicine, climate science, and fact-checking. In these fields, a high BERTScore or ROUGE score can mask catastrophic factual errors or hallucinations.
In medical report generation, Kaveri Kale et al. demonstrate in “Knowledge is Power”: Constructing Knowledge Graph of Abdominal Organs and Using Them for Automatic Radiology Report Generation that automated metrics completely miss clinical relevance and medical accuracy. This is supported by Kristine Ann M. Carandang et al. in Are LLMs reliable? An exploration of the reliability of large language models in clinical note generation, who show that while models might achieve high BERTScores for semantic consistency, human expert evaluation is mandatory to detect subtle, clinically dangerous inaccuracies.
Similarly, in climate science retrieval-augmented generation (RAG), David Thulke et al. find in Listen to the Context: Towards Faithful Large Language Models for Retrieval Augmented Generation on Climate Questions that while automated frameworks like RAGAS help track model faithfulness, they cannot fully validate real-world reliability without human cross-checking.
In fact-checking and misinformation, automated metrics are equally blind to human perception. Fengzhu Zeng notes in JustiLM: Few-shot Justification Generation for Explainable Fact-Checking of Real-world Claims that metrics like ROUGE and MAUVE fail to capture the journalistic nuances required for real-world claim justification. Furthermore, when evaluating the persuasiveness of AI-generated fake news, Chong Tian et al. argue in A Symbolic Adversarial Learning Framework for Evolving Fake News Generation and Detection that automated metrics cannot model how human psychology responds to misinformation, necessitating large-scale human studies.
The Pragmatic Bottlenecks of Human Evaluation
If automated metrics are so deeply flawed, why do they remain dominant? The literature points to severe pragmatic bottlenecks: cost, scalability, and a lack of standardized evaluation frameworks.
Shaoxiong Ji et al. explicitly highlight this in How Many Languages Make Good Multilingual Instruction Tuning? A Case Study on BLOOM, noting that budget constraints often force a total reliance on automatic metrics, potentially overlooking cultural appropriateness and fluency nuances.
To address the lack of standardization, some researchers have attempted to formalize human evaluation. Thomas Yu Chow Tam et al. propose the QUEST framework in A framework for human evaluation of large language models in healthcare derived from literature review. However, the authors note an inverse problem: by focusing exclusively on human assessment, the framework misses out on the scalability and cost benefits of automated metrics.
This tension suggests that the future lies not in abandoning automated metrics entirely, but in developing hybrid, human-in-the-loop systems. For instance, Marie Bexte et al. explore this balance in Scoring with Confidence? – Exploring High-confidence Scoring for Saving Manual Grading Effort, demonstrating how automated scoring can handle high-confidence cases while deferring ambiguous text to human experts. Similarly, Mingda Li et al. show in Unraveling and Mitigating Retriever Inconsistencies in Retrieval-Augmented Large Language Models that while automated voting mechanisms can optimize multi-source retrieval, incorporating human-annotated data is essential for truly calibrating answer trustworthiness.
Key Foundational Work
Designing a Symbolic Intermediate Representation for Neural Surface Realization (2019) — contributed 1 concept elements to this direction.
Relevant and Informative Response Generation using Pointwise Mutual Information (2019) — contributed 1 concept elements to this direction.
STAIR Captions: Constructing a Large-Scale Japanese Image Caption Dataset (2017) — contributed 1 concept elements to this direction.