Large Language Models (LLMs) are increasingly capable of passing professional ethics exams and identifying socially acceptable values in multiple-choice tests. However, when these same models are asked to generate open-ended text, they often fail to adhere to the very principles they just identified. This phenomenon, often called the "say-do gap," reveals a fundamental disconnect between a model's discriminative knowledge (the ability to recognize a value) and its generative behavior (the ability to act on that value).
The Illusion of Alignment in Static Benchmarks
Traditional evaluation of AI safety relies heavily on static, multiple-choice benchmarks. In these settings, a model might correctly select "Option B" as the most ethical response to a dilemma. However, as Shitong Duan, Xiaoyuan Yi, and their colleagues argue in DENEVIL: TOWARDS DECIPHERING AND NAVIGATING THE ETHICAL VALUES OF LARGE LANGUAGE MODELS VIA INSTRUCTION LEARNING, these benchmarks overestimate a model's actual safety.
The researchers found that models frequently fail to apply their internal ethical knowledge when responding to provocative or "red-teamed" prompts. This suggests that the model's "moral compass" is not a fixed property but a fragile state that can be bypassed during the decoding process. A model might "know" a rule in a vacuum but fail to prioritize it when navigating the complexities of creative text generation.
The Gen-Eval Gap: Why Models Fail to Practice What They Preach
If a model knows what a "good" response looks like, why doesn't it produce one? Inha Cha, Alice Oh, and their team explored this in Uncovering Factor Level Preferences to Improve Human-Model Alignment. They identified a "Gen-Eval gap": models are significantly better at judging human-preferred traits in existing text than they are at generating those same traits.
The intuition here is that during generation, models often fall back on "lazy" heuristics—such as prioritizing output length or surface-level fluency—rather than deep intent alignment or conciseness. Even if the model's internal reward mechanism recognizes that a shorter, more ethical response is better, the path of least resistance during token-by-token decoding often leads elsewhere. Alignment failures, therefore, are often not a lack of knowledge, but a failure of prioritization during the act of creation.
Mapping the Value Compass
To move beyond the limitations of multiple-choice tests, researchers have developed more dynamic evaluation frameworks. In Value Compass Leaderboard: A Platform for Fundamental and Validated Evaluation of LLMs Values, Jing Yao and Xiaoyuan Yi introduced a generative evaluation paradigm.
Instead of static questions, the Value Compass uses a self-evolving generator to create diverse scenarios that test how models navigate 27 different value dimensions. By analyzing the model's actual behavioral output across these dimensions, researchers can create a "value map" that more accurately predicts how a model will behave in real-world interactions. This approach confirms that high scores on discriminative tests do not guarantee value-consistent actions in open-ended scenarios.
Navigating and Patching Ethical Blind Spots
Understanding the gap is the first step toward fixing it. Two primary strategies have emerged from this research:
- Contextual Value Instructions: In the VILMO framework introduced by Shitong Duan, the model learns to generate tailored, context-specific "value instructions" for itself. Rather than relying on a generic safety prompt, the model identifies the specific ethical stakes of a prompt and uses that realization to guide its generation.
- Fine-Grained Factor Alignment: The PROFILE framework by Inha Cha decomposes human preferences into specific factors like intent and conciseness. By leveraging the model's superior ability to evaluate these factors, researchers can use self-guidance to bridge the gap, forcing the model to align its generative output with its own internal evaluative standards.
By treating ethical alignment as a problem of translation (from knowledge to action) rather than just acquisition (learning the rules), these methods provide a more robust path toward AI that truly follows human values.