Aligned language models appear to store their refusal behavior in a small set of internal units, and both attackers and defenders act on those units. Attackers suppress them, for example by forcing the answer to begin with "Sure, here is…". Defenders read them or push on them to restore refusals. Which units form that set is currently chosen by heuristics: scores computed one neuron at a time, a smoothness penalty on a graph of neurons, or a single linear probe. We propose a structural definition taken from graph algorithms. First build a graph whose nodes are safety-relevant neurons and whose edges join neurons that fire together. Then find the smallest set of "hub" neurons whose removal breaks the rest of the graph into shallow, star-like pieces. That hub set is called a treedepth modulator. Treedepth measures how many rounds of deleting one vertex per piece are needed before the graph falls apart completely, so a low value means no long chains of influence remain.
The claim is concrete and can fail. If the safety computation really is a switch controlled by a few hubs, then the modulator should behave as a causal bottleneck. Removing it should collapse refusal more than removing any other set of the same size. Reading it should give a cheap monitor that works while tokens are being generated. Freezing it during fine-tuning should protect safety with fewer frozen units than current methods need. If a simple ranking of neurons by how many connections they have does just as well, the structural lens adds nothing, and the experiments are built so that we find this out.
Core Claim
Write $G$ for the thresholded co-activation graph over safety-relevant units of one model. A set $S$ is a depth-$d$ modulator if
$$\mathrm{td}(G - S) \le d,$$
where $\mathrm{td}$ is treedepth and $d$ is a small constant, for example 3 to 5. We hypothesize three things. First, for aligned chat models, $G$ has a modulator much smaller than its vertex count. Second, $S$ is where refusal is decided, so interventions on $S$ dominate interventions on equal-sized sets chosen by degree, by per-unit attribution, or at random. Third, the remaining shallow components can each be summarized by a tiny per-component readout, the way a kernel summarizes a piece of a graph. A monitor built from modulator activations plus these summaries then matches full-hidden-state probes at lower cost and without injecting extra tokens.
Where This Comes From
The method comes from parameterized complexity. Kernelization Using Structural Parameters on Sparse Graph Classes shows that once a graph is split into a small treedepth modulator and constant-treedepth pieces attached to it through small boundaries, each piece can be replaced by a constant-size equivalent gadget without changing the answer. Our per-component summaries are the empirical analogue of those gadgets, and we claim no formal equivalence. Where First-Order and Monadic Second-Order Logic Coincide proves that bounded treedepth is exactly the condition under which global properties reduce to local ones on classes closed under subgraphs. This is why we pick treedepth rather than treewidth. A long path has treewidth 1 but treedepth growing with its length, so only a treedepth bound rules out long relay chains through which a suppressed signal could reach the output around the hubs. That same fact is a constraint. The file notes that treedepth modulators can be much larger than treewidth modulators on long, thin graphs, so a large modulator is a real possible outcome. Computing treedepth exactly is NP-hard, and exact PACE 2020 solvers handle only about 170 vertices (PACE 2020 Treedepth, PID⋆). This limit shapes the algorithm below.
Two safety papers motivate the proposal. Superficial Safety Alignment Hypothesis finds that safety-critical units make up about 1.3–1.4% of units and act as a binary refuse-or-fulfill switch. It selects them one unit at a time, whereas we select them jointly by graph structure. Graph-Regularized Sparse Autoencoders for LLM Safety Steering already builds a neuron co-activation graph but uses it only as a Laplacian smoothness penalty, which pushes steering directions to vary slowly across connected neurons. It never decomposes the graph, and its steering bank falls to about 52% refusal under white-box attack. We reuse its graph construction and replace the spectral penalty with a combinatorial decomposition.
The inference-time comparison point is Any-Depth Alignment. Its linear probe (ADA-LP) reads hidden states after injecting the assistant header token mid-generation. Its "depth" is token position in the sequence, which is unrelated to our structural depth. Backtracking Improves Generation Safety and Reinforcement Learning with Backtracking Feedback provide the recovery action our monitor would trigger. We deliberately do not formalize backtracking as an elimination tree. That tree's depth is simply the number of backtracks, and a treedepth framing would add vocabulary without content.
Research Program
Graph construction. For Llama-3-8B-Instruct and Llama-2-7B-Chat, we first restrict to a few thousand candidate units ranked by attribution to the refusal logit, measured on contrasting harmful and benign prompts from AdvBench and XSTest. We then connect two units when their co-firing, measured by pointwise mutual information, exceeds a threshold on a mixed prompt corpus. Results are reported across a sweep of thresholds so that no single choice drives them.
Modulator search. The search runs in three steps. It greedily removes the vertex that most reduces a heuristic treedepth upper bound, taken from DFS-tree depth. Once components fall below about 150 vertices, it certifies their treedepth exactly with PID⋆. A local-search step then tries to drop vertices from $S$ while keeping every component within the depth bound. This produces modulator size as a function of $d$, a quantity of interest in its own right.
Three uses of the modulator. The monitor is a logistic probe on modulator activations plus one scalar readout per shallow component, evaluated at every token with no token injection. The steer adds activation to modulator units when the monitor fires, or alternatively triggers a backtracking reset. The shield freezes modulator units during benign fine-tuning, which is a direct counterpart to SSAH's freezing of its safety-critical units.
Baselines
The naive reference is the unmodified aligned model. Under deep prefill it refuses at close to 0% (Any-Depth Alignment), and GCG succeeds 62.5% of the time on Llama-3-8B (Backtracking).
The strongest monitoring competitor is ADA-LP. On the deep-prefill suite it reports near-100% refusal, a near-0% false-positive rate across benign suites including XSTest, and about 25 ms per check. We rerun it on the same models, prompts, and probe-training data as our monitor, and we tune its probe interval and regularization with the same search budget. Our monitor beats it if it matches its refusal and false-positive rates while checking every token rather than periodically, which closes ADA's admitted leakage window between checks, at equal or lower latency. Matching its refusal rate with fewer tokens leaked before halting also counts as a win.
The strongest steering competitor is GSAE. It reports selective-refusal gains of 20.1 points on JailbreakBench and 16.8 on HarmBench over a standard sparse autoencoder on Llama-3-8B, 170.9 tokens per second against 208.3 unsteered, and roughly 71% and 52% refusal under gray-box and white-box attack. We use its co-activation graph as our starting point so that the only difference is the decomposition.
The strongest shielding competitor is SSAH. On Llama2-7B fine-tuned on benign data it reports 18.4% AdvBench attack success, and 7.88% when its safety-critical units plus the top 6% of complex units are frozen. Our shield beats it if a smaller frozen set reaches equal or lower attack success.
The ablations isolate the structural ingredient. Each uses the same unit budget $|S|$ drawn from the same candidate pool: top-degree units, top-betweenness units, SSAH-score units, a treewidth modulator (feedback-vertex-set style), and random units.
Why not just…?
Why not just take the highest-degree neurons? This is the most serious threat. On graphs built around a few dominant hubs, a greedy modulator and a top-degree set largely coincide. We will measure how much the two sets overlap. We will also report the causal comparison separately on units that are in the modulator but not in the top-degree set, and on units in the top-degree set but not in the modulator. If knocking out the modulator-only units does not hurt refusal more than knocking out the degree-only units, we will conclude that treedepth adds nothing beyond centrality and report that.
Why not just use SSAH's per-unit scores? Scoring units one at a time can miss units that matter only because they bridge clusters, and can include redundant units that sit inside one cluster. The shielding experiment settles this directly against the 7.88% figure. If SSAH's set is already minimal, our modulator will not be smaller, and we will say so.
Why not just run ADA's linear probe on the full hidden state? It already works extremely well on its reported suites, so detection accuracy is not where we expect to win. The possible gains are per-token checking without header injection, and a smaller, interpretable readout that adaptive attacks may find harder to evade. If a full-state probe evaluated at every token has equal cost and equal robustness, our monitor has no advantage, and the value of the proposal rests on the causal and shielding results.
Why treedepth rather than a treewidth modulator? Treewidth allows long thin chains to remain after removing the hubs, and such chains are exactly the relay routes our causal story rules out. The treewidth-modulator ablation tests this. If it performs equally well at equal size, the "no long chains" argument is wrong.
Decisive Experiments
The first check is structural. On Llama-3-8B-Instruct, is the depth-4 modulator small, for example under 1.5% of candidate units, and stable across thresholds and random seeds? If not, the hypothesis fails at the first step.
The second check is causal. Ablating $S$ should raise AdvBench and JailbreakBench attack success, judged as in JailbreakBench, more than any equal-size baseline set. Steering $S$ under deep prefill should restore refusal to near ADA-LP's level while keeping XSTest safe-prompt refusal close to the unsteered model and MMLU and GSM8K within a point.
The third check is fine-tuning. Freeze $S$ during Alpaca and Dolly fine-tuning and compare AdvBench attack success against SSAH's 7.88%, reporting the size of the frozen set.
The fourth check is adversarial. We run GCG and white-box attacks that know $S$ and the monitor, and compare against GSAE's ~52% white-box refusal, with no advance claim of winning.
We also report throughput against the 208.3, 170.9, and 45.1 tokens-per-second reference points. A clear negative result on the causal check, meaning degree-ranked sets matching the modulator, would be worth publishing as evidence that safety circuits are organized around hubs but not specifically shallow once the hubs are removed.
References
- Kernelization Using Structural Parameters on Sparse Graph Classes
- Where First-Order and Monadic Second-Order Logic Coincide
- The PACE 2020 Parameterized Algorithms and Computational Experiments Challenge: Treedepth
- PACE Solver Description: PID^⋆
- Superficial Safety Alignment Hypothesis
- Graph-Regularized Sparse Autoencoders for LLM Safety Steering
- Any-Depth Alignment: Unlocking Innate Safety Alignment of LLMs to Any-Depth
- Backtracking Improves Generation Safety
- Reinforcement Learning with Backtracking Feedback