Hierarchical Taxonomies for Fine-Grained AI Safety

151 concepts105 papers1 proposal

Imagine a content moderation system for an AI assistant that only has two options: mark a response as "safe" or "unsafe." If a user asks for detailed information about a rare medical condition, a binary system might panic and block the query to avoid liability, leaving the user empty-handed. Conversely, it might allow the response, inadvertently providing unauthorized medical advice.

When safety is treated as a simple binary boundary, models suffer from an "alignment tax"—they become overly evasive, unhelpful, or prone to catastrophic blind spots because they cannot differentiate a minor social faux pas from a severe threat. As noted by Gavin Abercrombie et al. in Risk-graded Safety for Handling Medical Queries in Conversational AI, flat binary classifications fail to capture the nuanced spectrum of risk inherent in complex domains. To build models that are both highly capable and robustly secure, the research community is shifting toward hierarchical and multi-level risk taxonomies.

Deconstructing Harm: The Hierarchical Architecture

Instead of drawing a single line between acceptable and unacceptable content, a hierarchical taxonomy organizes safety into structured layers. Typically, this involves a top-down decomposition moving from broad risk domains down to granular, actionable categories.

For example, Lijun Li et al. in SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models structure safety evaluations across distinct tiers: domains, tasks, and specific categories. In an image generation context, a similar multi-level approach allows automated guardrails to provide specific, structured justifications for a violation rather than a blunt, uninformative rejection (Lijun Li et al., T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generation).

[Broad Domain]  --->  [Specific Task/Hazard]  --->  [Granular Category]
(e.g., Malicious Use)   (e.g., Cyberattacks)          (e.g., Social Engineering Code)

This vertical depth is often paired with horizontal dimensions that evaluate the severity of the potential harm and the intent of the user. Giulio Zizzo et al. demonstrate in HarmLevelBench: Evaluating Harm-Level Compliance and the Impact of Quantization on Model Alignment that categorizing prompts into discrete levels of severity allows developers to map out a model's exact "breaking point"—the threshold where its safety alignment begins to degrade under pressure.

Why Granularity Improves Alignment

Moving from binary labels to a multi-level taxonomy is not just an exercise in data labeling; it fundamentally changes how safety models learn and behave.

1. Richer Optimization Signals

When training a model via Reinforcement Learning from Human Feedback (RLHF), a binary reward signal ("good" vs. "bad") provides very little structural information. In contrast, training on a highly granular taxonomy provides a dense optimization landscape. As argued by Shaona Ghosh et al. in AEGIS2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrails, a detailed categorization gives the guardrail a richer understanding of safety boundaries. This allows the model to better navigate borderline cases and generalize to entirely new, unseen risks at inference time.

2. Eliminating Annotator Ambiguity

Human annotators frequently disagree on whether a prompt is "toxic" or "unsafe" because these terms are highly subjective. By breaking down risks into explicit, multi-level definitions, frameworks drastically reduce annotator inconsistency (Jiaming Ji et al., Safe RLHF-V: Safe Reinforcement Learning from Multi-modal Human Feedback). When annotators evaluate specific dimensions—such as distinguishing between a biased attitude, a discriminatory act, or an explicit threat of violence—the variance in human labels drops, leading to cleaner training data (Mohit Chandra et al., AbuseAnalyzer: Abuse Detection, Severity and Target Prediction for Gab Posts).

3. Context-Sensitive Moderation

Many safety failures are context-dependent. A phrase like "I agree" is completely benign on its own, but dangerous if it follows a user prompt promoting hate speech or dangerous medical misinformation. Howe Tissue et al. show in On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark that mapping responses to specific categories like "biased agreement" allows guardrails to catch context-sensitive failures that general-purpose, flat safety tools completely miss.

4. Uncovering Hidden Model Inclinations

Binary evaluations often mask a model's underlying vulnerabilities. If a model provides a neutral or moderately risky response to a dangerous prompt, a binary classifier must make a forced choice, often marking it as "safe." Single-author work by Ling Shi in CRiskEval: A Chinese Multi-Level Risk Evaluation Benchmark Dataset for Large Language Models demonstrates that a multi-level grading system exposes these subtle hazardous tendencies—such as a model's latent willingness to deceive or seek power—long before they manifest as overt safety failures.

Grounding Taxonomies in Real-World Standards

To ensure these multi-level frameworks are not just arbitrary academic constructs, researchers are increasingly grounding them in established external structures.

One prominent approach is to derive taxonomies directly from legal and corporate compliance standards. For example, Yi Zeng et al. developed AIR-BENCH 2024: A Safety Benchmark based on Regulation and Policies Specified Risk Categories by decomposing 8 global government regulations and 16 corporate acceptable use policies into a massive, 314-category hierarchy. This bridges the gap between raw model outputs and actual legal compliance.

Other frameworks adapt traditional industrial safety engineering to the AI landscape. In A Hierarchical HAZOP-Like Safety Analysis for Learning-Enabled Systems, Yi Qi et al. stratify safety analysis into three distinct layers: the overall System, the ML-Lifecycle, and the Inner-ML model components. This hierarchical structure allows engineers to trace exactly how a low-level neural network anomaly propagates upward into a high-level system failure.

By moving away from binary classification and embracing these structured, multi-tiered risk profiles, developers can deploy guardrails that are flexible, legally compliant, and capable of nuanced refusal behaviors without sacrificing the model's core utility.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_binary_safety_classification_0991c7b5,
  title = {binary safety classification},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/binary-safety-classification/rdr_0991c7b5b3574d6fa794f7e878c60079},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_binary_safety_classification_0991c7b5_v1,
  title = {binary safety classification},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/binary-safety-classification/rdr_0991c7b5b3574d6fa794f7e878c60079/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake