Imagine a security guard protecting a secure facility. To evaluate the guard's effectiveness, an auditor tests them by walking up and asking for the keys. If the guard says "No," the auditor marks it as a success. If the guard says "Sure, here you go," but hands over a plastic toy key that doesn't fit any lock, traditional AI safety benchmarks would still flag this as a catastrophic security failure.
This is the core pathology of early Large Language Model (LLM) safety evaluation. For years, the standard way to measure whether a model had been "jailbroken" was binary keyword matching: did the model output a refusal phrase like "I cannot fulfill this request", or did it start with an affirmative phrase like "Sure, here is how to..."?
A growing body of research demonstrates that this binary approach creates a profound illusion of both vulnerability and safety. Instead, a new paradigm is emerging: actionability- and utility-based safety evaluation. This framework shifts the focus from surface-level textual compliance to the actual real-world utility and technical accuracy of the generated content.
The Anatomy of the Metric Gap
Binary safety metrics suffer from two symmetrical flaws: they overestimate risk by counting harmless gibberish as successful attacks (false positives), and they underestimate risk by missing subtle, fragmented, or implicit safety violations (false negatives).
False Positives: The Compliance Illusion
When optimization algorithms or creative prompters attack an LLM, they often optimize for the path of least resistance: forcing the model to bypass its initial refusal trigger. As Sicheng Zhu et al. demonstrate in AdvPrefix: An Objective for Nuanced LLM Jailbreaks, attacks frequently succeed at eliciting an affirmative opening phrase without actually forcing the model to generate a coherent or faithful harmful response.
When evaluated by simple dictionary-based string matching, these responses are flagged as successful jailbreaks. However, the authors of Geneshift: Impact of different scenario shift on Jailbreaking LLM point out that these metrics are easily misled by outputs that are entirely nonsensical, irrelevant, or benign gibberish, leading to heavily inflated attack success rates.
Furthermore, models frequently hallucinate when pushed past their safety boundaries. Lingrui Mei et al. address this explicitly in "Not Aligned" is Not "Malicious": Being Careful about Hallucinations of Large Language Models' Jailbreak, introducing a distinction between a model that is merely "not aligned" (failing to refuse) and one that is truly "malicious" (providing functional, dangerous information). Without verifying whether code is executable or factual information is accurate, traditional benchmarks mistake harmless hallucinations for genuine real-world threats.
False Negatives: The Subtle Leak
Conversely, sophisticated adversarial strategies can extract dangerous knowledge without ever triggering a binary safety filter. For instance, David Glukhov et al. introduce the concept of inferential leakage in Breach By A Thousand Leaks: Unsafe Information Leakage in 'Safe' AI Responses. By decomposing a dangerous query into multiple seemingly benign sub-questions, an attacker can piece together hazardous instructions from a series of "safe" responses. Because each individual interaction passes the binary refusal check, traditional evaluations register zero risk, completely missing the cumulative hazard.
Similarly, Yao Huang et al. show in Breaking the Ceiling: Exploring the Potential of Jailbreak Attacks through Expanding Strategy Space that implicit safety violations—where a model aligns with a harmful intent without using flagged keywords or obvious compliance patterns—routinely evade standard binary metrics.
Grounding Safety in Actionability and Utility
To fix this metric gap, researchers are developing frameworks that evaluate the quality-of-service of the jailbreak from the perspective of a malicious actor. If a jailbreak occurs, how much closer does it actually bring an adversary to executing a physical or digital threat?
[Traditional Metric] --> Did the model say "Sure"? --> Binary Success/Failure
│
▼ (Flawed Proxy)
[Modern Paradigm] --> Is the content accurate? ──┐
Is the content detailed? ──┼─> Actionable Utility Rate
Is it physically doable? ──┘
The Actionability Framework
Instead of checking for a binary state, recent frameworks score the generated text along two primary axes: actionability (the step-by-step feasibility of the instructions) and informativeness (the depth and specificity of the underlying knowledge).
Yik Siu Chan et al. formalize this in Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactions, using the geometric mean of actionability and informativeness to quantify the practical utility of a response. This metric ensures that a response is only considered dangerous if it provides actionable assistance for high-stakes threats (such as chemical synthesis or illegal acts), rather than just vague or high-level descriptions.
To capture these nuances, Haoming Yang et al. propose a ranking-based evaluation framework in Cannot See the Forest for the Trees: Invoking Heuristics and Biases to Elicit Irrational Choices of LLMs. Instead of assigning a flat pass/fail, outputs are ranked by their severity and practical utility, allowing defenders to separate low-risk alignment failures from high-risk actionable instructions.
The Capability-Utility Composite
Evaluating compliance in a vacuum also ignores the underlying intelligence of the model. A small, poorly trained model might have a 100% compliance rate on harmful prompts but lack the technical capacity to generate a working cyberattack exploit. Conversely, a highly capable frontier model might have a 99% refusal rate, but the remaining 1% of successful breaches could yield highly optimized, devastatingly accurate instructions.
To capture this dynamic, Ann-Kathrin Dombrowski et al. developed The Safety Gap Toolkit: Evaluating Hidden Dangers of Open-Source Models. They calculate a model's effective dangerous capability by multiplying its technical knowledge accuracy in high-risk domains by its compliance rate. This approach reveals a hidden risk as models scale: larger models are often better at refusing prompts out of the box, but they harbor far more dangerous, highly actionable knowledge if an attacker successfully bypasses their guardrails.
This paradox is further explored by Chengda Lu et al. in Does Chain-of-Thought Reasoning Really Reduce Harmfulness from Jailbreaking?. By utilizing comparative evaluation metrics that specifically isolate the level of technical detail, the authors demonstrate that while advanced reasoning models are harder to jailbreak, their internal Chain-of-Thought (CoT) mechanisms actually make their responses significantly more dangerous and actionable once a breach finally occurs.
Why This Paradigm Shift Works
Moving from surface-level keyword matching to semantic utility evaluation works because it aligns the evaluation metric with the actual threat model.
- It filters out optimization artifacts: Automated attacks often exploit the token-prediction mechanics of LLMs to force affirmative tokens. Utility-based judges look past the initial tokens to evaluate the semantic payload of the entire response, neutralizing false positives generated by adversarial gibberish.
- It leverages advanced semantic judges: Rather than relying on rigid regular expressions, modern frameworks use advanced LLM-based judges (like GPT-Judge) or specialized domain-expert models. As noted by Hua Tang et al. in GRAF: Multi-turn Jailbreaking via Global Refinement and Active Fabrication, these advanced judges can successfully distinguish between a model that avoids refusal keywords but still fails to fulfill the request, and one that genuinely provides harmful utility.
- It measures real-world risk: By grounding safety in whether an output is accurate, complete, and actionable, the AI safety community can move away from chasing superficial alignment behaviors and focus on mitigating genuine, high-stakes capabilities that pose real-world dangers.
Key Foundational Work
Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactions (2025) — contributed 5 concept elements to this direction.
Geneshift: Impact of different scenario shift on Jailbreaking LLM (2025) — contributed 4 concept elements to this direction.
JPS: Jailbreak Multimodal Large Language Models with Collaborative Visual Perturbation and Textual Steering (2025) — contributed 4 concept elements to this direction.