Copy-or-Compute Reasoning Traces: Template-Constrained Chain-of-Thought That Cannot Leak What It Was Not Shown

When a language model "thinks out loud" before answering (chain-of-thought prompting), the intermediate text becomes a new channel for leaking memorized training data. Recent audits show this channel can stay open even after the model has supposedly forgotten the data. We propose closing it by construction, using an old idea from natural language generation: templates. Every reasoning step is restricted to a fixed sentence skeleton with typed slots. Each slot may hold only one of two things. The first is a span copied from the user's input. The second is a value computed by an external checker from earlier slots, such as the result of an arithmetic step. The model chooses which skeleton to use and where to copy from, but it never writes free text. A reasoning trace built this way cannot spell out a phone number, an author biography, or a copyrighted passage that does not appear in the prompt.

The opportunity matters for three reasons. Reasoning traces are logged, stored in agent memory, and passed to tools as they are produced. Today's defenses against leakage through traces are learned behaviors that come with no guarantees. And the evidence shows that what a trace contains shapes the final answer, so cleaning the trace after the fact is too late. The open question is whether such a restricted trace can still do useful reasoning. It is also open whether blocking the trace starves the leak in the answer as well, or only moves it there.

Core Claim

Our claim is that a reasoning trace generated under a copy-or-compute template grammar has a small, computable upper bound on how much information it can carry that is not already in the input. We further claim that, in practice, this bound drives leakage of memorized training data in the trace to near zero, cuts leakage in the final answer substantially, and does so at a modest accuracy cost on structured reasoning tasks.

The bound comes from counting the model's free choices. Let a trace have $S$ steps. At each step the model picks one of $T$ templates and fills its copy slots, each with a pointer into an input of length $L$ tokens, using at most $c$ copy slots per template. Computed slots are deterministic functions of earlier slots, so they carry zero free bits. The number of bits the trace can carry beyond the input is then at most

$$ B \le S\,\big(\log_2 T + c\,\log_2 L^2\big), $$

where the $L^2$ term counts possible start and end positions of each copied span. This is a capacity bound, not a leakage rate. A model could in principle hide a secret in its sequence of template choices. But a ten-digit phone number would have to be encoded on purpose across many steps. Nothing in pretraining rewards that. Unconstrained traces, by contrast, can reproduce such a string verbatim in a single step. We will measure how far below $B$ actual leakage falls.

The idea is clearly different from the closest work. Chain-of-Sanitized-Thoughts trains models with fine-tuning and placeholder prompts ([PERSON], [ADDRESS]) to "think privately." That is a learned behavior that only targets entity types defined in advance, and its authors note it is unverified against extraction attacks. SALT pushes the model's internal activations away from a "leak" direction at inference time, again without a guarantee. Our constraint is enforced by the decoder. It is agnostic to entity type, because it blocks anything absent from the input rather than a list of PII categories.

Research Program

The template side reuses the corpus-driven tradition in generation. Novais et al. built a "select-and-modify" system: canned sentence skeletons with typed Agent/Action/Patient slots, extracted from email corpora. Scirea et al. mined part-of-speech sequences from a reference corpus as structural templates. We apply the same move to reasoning traces.

The first step is to collect traces from the target model on clean, PII-free task data. We then cluster the steps by their non-content tokens and induce a library of skeletons. A step like "Subtract [copy:quantity] from [copy:quantity] to get [compute:sub]" is one example. Slots are typed as copy (pointer into the input), compute (checked by a calculator or a simple symbolic executor), or closed-class (a small enumerated set such as yes/no or entity-type labels). The library is induced per task family: math word problems, reading or extraction, and question answering over biographies.

The second step is to decode under the induced grammar. We use sketch-aware decoding from Prompt Sketching, which treats fixed text and multiple placeholders as one jointly decoded object with variable-level beam search. We extend it with pointer-constrained slot filling and checker-filled compute slots. Fixed skeleton text is forced, which echoes how TORSO forces reasoning delimiters. The answer channel is decoded freely in the main condition. A secondary condition applies the same copy-or-compute constraint to answers where the task allows it.

The third step is optional and lightweight. A short fine-tune on the model's own clean traces, rewritten into template form, teaches it the grammar so that constrained decoding is not fighting the model's preferences. LiteCoST showed something analogous: fine-tuning plus reinforcement learning on format and process rewards let 3–7B models reason through structured intermediate tables with large gains on Loong Finance. We will report results with and without this step. That way the guarantee, which depends only on decoding, is never confused with a learned effect.

The analysis has two strands. On the utility side we use the token-level memorization attribution of STIM. STIM found that local n-gram memorization drives about 67.3% of applied-math reasoning failures on OLMo 2. We ask whether template traces, which forbid free n-gram continuation, remove these failures or merely trade them for template-selection errors. This is a secondary hypothesis, not the main claim.

On the leakage side we address a known confound. Luo et al. showed that apparent PII memorization collapses to near zero once prompt-target overlap is controlled. Many "hits" are cue artifacts, such as guessing [email protected]. We therefore report all extraction results stratified by their overlap score. Copy slots make this especially important: if a target string partly appears in the prompt, copying it is not memorization.

We make no claim about parameter capacity. Yu, Gao, and Zhang prove that chain-of-thought and direct prediction both need on the order of $N$ parameters to memorize $N$ examples. Our method does not remove memorized data from the weights; it limits where that data can surface.

Baselines

The naive reference point is the same model with ordinary unconstrained chain-of-thought and no privacy measure. It is run on identical prompts, with identical sampling settings and the same number of attack queries. Every other method is scored against it.

The strongest learned competitor for leakage through the trace is supervised fine-tuning from Chain-of-Sanitized-Thoughts. On PII-CoT-Bench it reports a 98.8% reduction in Total Leakage Rate for GPT-OSS-20B, with utility typically above 95/100. We will retrain it on the same base models, with the same LoRA budget we give our optional template fine-tune, on its released 1,200 training examples.

Beating this baseline means two things. On parametric (training-data) leakage, where its placeholder scheme is not designed to apply, we must show strictly lower trace and answer leakage. On PII-CoT-Bench itself we must match it within a few points, using a copy rule that admits PII-tagged input spans only as placeholders. We will not claim to beat a 98.8% reduction on that benchmark. We will claim to generalize beyond it.

The strongest inference-time competitor is SALT. It reports Contextual Privacy Leakage reductions of 31.2% on DeepSeek-R1-Distill-Qwen-1.5B, 18.2% on QwQ-32B, and 17.9% on Llama-3.1-8B on AirGapAgent-R. We run it at its recommended late-layer injection, with steering vectors built from the same number of leak/non-leak pairs we use to induce templates. On parametric leakage there is no reported result found on Lacuna for SALT, so we will establish one ourselves.

For verbatim copying of training data, the strongest decoding-time competitor is Anchored Decoding. It fuses the risky model's next-token probabilities with those of a small model trained only on permissively licensed text, and reports closing up to 75% of the copying gap. We apply it to the trace and answer together, with its reference model. This comparison matters because anchored decoding also limits what can be emitted, but softly and without a capacity bound.

Two ablations isolate the key ingredient. The first, vocabulary-only, keeps free-form traces but restricts tokens to those in the input plus a fixed list of function words and digits. This tests whether templates and checked compute slots matter beyond a simple token whitelist. The second, template-free-slots, keeps the skeletons but lets slots be filled freely. This tests whether the copy-or-compute rule, not the skeleton, does the work.

Why not just…?

Why not just hide or drop the reasoning trace? This removes leakage through the trace but not through the answer, and it gives up the accuracy that reasoning provides. In agent settings the trace is often consumed by tools and memory even when users never see it, as SALT and Chain-of-Sanitized-Thoughts both note. Our no-trace condition will report both quantities, answer leakage and accuracy. If constrained traces do not beat no-trace on accuracy, the proposal fails on utility.

Why not just filter the trace after generation, deleting anything not found in the input? This is the most serious threat. It enforces nearly the same content rule as ours. The difference is causal. The answer is generated while conditioned on the unfiltered trace. The head-conditioned canary audit Auditing Reasoning-Trace Memorization Claims shows the trace drives the answer: after unlearning, answer accuracy on canaries was 0.60, and replacing the trace with a fixed prefill dropped it to 0.37. A leak that enters the trace therefore propagates into the answer before any filter runs. Post-hoc parsing is also fragile: missing </think> tags flipped the measured bypass gap to −0.92 on a Llama-8B distill. We run the filter as a full baseline. Its trace leakage should match ours by construction, so the decisive comparison is answer leakage. If the filter matches us there, our contribution shrinks to accuracy and auditability, and we will say so.

Why not just unlearn the data with Negative Preference Optimization (NPO), which lowers the model's probability on the forget set relative to a reference model? The canary audit applied NPO to answers and still found trace leakage of 0.83, a +0.23 gap over the answer, with teacher-forced top-1 match above 0.90. Wang et al. show that "forgotten" records are recoverable with light fine-tuning. Unlearning is complementary, so we also test NPO plus our decoder. We expect that combination, not either alone, to give the lowest answer leakage.

Why not just a system-prompt instruction to keep reasoning private? Chain-of-Sanitized-Thoughts found prompt-only privacy reasoning unreliable on DeepSeek-R1-Qwen-7B and vulnerable to manipulation, and Anchored Decoding found system prompts dominated by constrained decoding. We include it as a cheap row and expect it to be the weakest defense.

Proposed Experiments

The primary test uses the head-conditioned canary suite from the canary audit. In that suite, 60 fictional author biographies are force-memorized into DeepSeek-R1-Distill-Qwen-7B and a Llama-8B distill, and NPO is applied to answers. We reproduce the setup and measure trace leakage, answer leakage, and the bypass gap under each decoder. We score the trace from the decoder's own slot structure rather than tag parsing, which sidesteps the parser failure that paper documents. The decisive outcome is trace leakage near zero for our decoder, with answer leakage below the post-hoc filter at matched accuracy on the retained, non-canary questions.

The second test is prefix-grounded extraction on the Enron subset of the Pile, following PII-Compass. There, manual templates extract under 0.15% of phone numbers, while grounding with true prefixes from other data subjects reaches 6.86% at 2,308 queries. We wrap the extraction in a reasoning task: the model must reason about the email before answering. We then run the grounded attack against each decoder on a Pile-trained model, at query budgets of 1, 128, and 2,308, with results stratified by the overlap scoring of Luo et al.. This answers an open question in its own right: does reasoning raise or lower extraction compared with direct prompting? It also tests whether constrained traces push extraction back toward the template floor.

The third test probes adversarial pressure on the template-choice channel, the one place the capacity bound leaves room. Following the suffix-attack style of Excessive Reasoning Attack, we optimize adversarial suffixes against our decoder, aiming either to induce long traces or to steer template choices toward encoding a target digit string. We report measured leaked bits alongside the bound $B$.

Utility is measured on GSM8K and the STIM math, counting, and formula tasks, where we also count memorization-attributed failure points. We add Loong Finance against LiteCoST's reported gains, and PII-CoT-Bench and AirGapAgent-R for the contextual-privacy comparisons above.

The proposal is falsified if either of two things happens. One is that the post-hoc copy filter matches our answer leakage at equal accuracy. The other is that constrained traces lose more GSM8K accuracy than dropping the trace entirely.