Attestable Pedagogical Invariants: Which Tutoring Rules Can a Hardware Enclave Actually Guarantee?

AI tutors fail in a specific, measurable way: they hand students the answer. On math tutoring dialogues, ChatGPT revealed the final solution in 66% of its turns, while a small model fine-tuned for tutoring did so in only 1–4% of cases (MathDial). A school buying a hosted tutor has no way to check which of these it is getting, and the vendor has a cost incentive to cut corners, because step-by-step hinting takes more model calls than simply telling. A trusted execution environment (TEE) can help here. A TEE is a hardware-isolated region of a processor that even the machine's owner cannot read, and it can produce a signed "attestation" proving exactly which code ran inside it. Proof-of-Guardrail already uses this to prove that a safety filter ran on an AI agent's outputs (Proof-of-Guardrail). That work also names the catch: proving a filter ran does not prove the output is safe, because a developer controlling the prompt can jailbreak a learned filter from inside the enclave.

This proposal turns that catch into the research question. We split pedagogical rules into two kinds. The first kind are rules a teacher can state as a deterministic check against sealed teacher-authored material, for example "the numeric answer, or anything that evaluates to it, is not released before the student has completed step 3 of this solution tree." The second kind need a learned judge, such as "this hint is appropriate for the student's level." Our claim is that only the first kind yields a guarantee that survives an adversarial vendor. The enclave should therefore enforce teacher-authored invariants of the first kind, and issue signed multi-turn compliance reports that say, separately and honestly, how far the second kind can be trusted. The two directions play different roles. Teachers' domain knowledge supplies the policy. The enclave supplies the reason anyone outside the vendor can believe that policy was applied.

Core Claim

A small gate running inside an enclave can make answer leakage verifiable against an adversarial model provider. The gate holds a teacher-authored step tree and answer key, runs a deterministic check that includes symbolic and numeric equivalence, and tracks dialogue state. We expect the residual leak rate to be bounded by the gate's evasion rate under red-teaming, and that rate should be far lower for deterministic invariants than for LLM-judge guardrails. The gate is small enough to fit easily within enclave memory limits. The contribution is not "Proof-of-Guardrail for education." It is a measured map of which pedagogical properties remain meaningful under attestation when the party writing the prompts is the adversary, plus a mechanism that enforces the robust ones across whole sessions rather than single responses.

Research Program

The system has three parts. The first is policy authoring. Teachers build a step tree for each problem, meaning an ordered decomposition of the solution with a hint ladder at each node. The design follows DBox's interactive step trees for algorithmic problems (DBox), Michener's argument that tutoring needs explicit auxiliary structure such as prerequisites and heuristics beyond the bare solution (Structuring Mathematical Knowledge), and the no-code "teacher as playwright" interface of Co-Pilot (A Piece of Theatre). Co-Pilot reported that teachers fear LLM unpredictability once a bot is unmonitored, and this project responds to that fear directly. The policy is encrypted to the enclave, so the vendor never sees answer keys or class material.

The second part is the gate. It is an open-source wrapper, measured by attestation as in Proof-of-Guardrail. It sits between the vendor's model, which runs in a confidential VM, and the student. For every candidate response it runs two groups of checks. The first group are invariants: extraction and symbolic evaluation of any expressions against the sealed answer and intermediate values, and enforcement of step order against the dialogue state. The second group are soft scores: a small intent classifier trained on the 11-intent taxonomy of Intent Matters, and a LoRA rubric judge in the style of AIriskEval-edu. A response that violates an invariant is blocked and the vendor is asked to regenerate. Soft scores are logged but do not carry the guarantee.

The third part is certification. At session end, and over rolling windows, the enclave signs a report containing invariant violations blocked, regeneration counts, the intent distribution, and soft-score summaries. It binds these to the policy hash and the gate hash, and releases the aggregate without releasing transcripts. This extends pedagogical auditing from single explanations, which is AIriskEval-edu's stated limitation, to multi-turn trajectories.

The central analysis is adversarial. We play the vendor with full control over the system prompt and model, and with the goal of leaking the answer while passing the gate. Attacks include paraphrase ("two less than ten"), arithmetic encoding, splitting the answer across turns, answering the final step disguised as a hint, and prompt injection aimed at the soft judge. For each property we report the evasion rate. This produces the taxonomy of attestable versus merely logged properties.

Baselines

The naive reference point is a general instruction-tuned model prompted to tutor Socratically, with no gate. MathDial reports ChatGPT leaking the solution in 66% of turns and scoring 1.43 on Equitable Tutoring on a 3-point scale (MathDial). We rerun a current open model under the same prompt on the same problems so that the numbers are directly comparable.

The strongest behavioral competitor is a tutor fine-tuned for restraint: the MathDial Flan-T5 780M model (reported 1–4% leakage, 2.19 Equitable Tutoring) and a Mistral-7B tuned on the 11-intent taxonomy (preferred 56.7% of the time over the 4-intent version in Intent Matters). This baseline is strongest because it achieves low leakage honestly. It offers no guarantee, however, when the vendor swaps it out or prompts it adversarially. We evaluate it in two conditions, cooperative and adversarial-vendor, using the same problems and the same leak annotation. Beating it means matching its cooperative leakage within noise while keeping leakage near our measured evasion floor in the adversarial condition, where an unverified fine-tuned model has no floor at all.

The strongest existing verifiable method is Proof-of-Guardrail with a learned guardrail. We run it both with Llama Guard, whose reported unsafe-content F1 on ToxicChat was 0.56 in Proof-of-Guardrail, and, more fairly, with the same LLM leak judge we use for soft scores. Both get the same enclave, the same red-team budget, and the same compute. Beating it means a significantly lower adversarial evasion rate on answer leakage at comparable latency.

Two ablations isolate the key ingredients. Replacing the deterministic invariant with the LLM judge, with everything else held equal, isolates determinism. Running the identical gate outside the enclave as a school-side proxy isolates what the hardware adds. The hardware should add nothing to leak prevention and should add only verifiability, vendor-blind policies, and transcript privacy. We predict exactly this, and the ablation exists so the paper cannot overclaim.

Why not just…?

Why not just use a strong system prompt or a fine-tuned tutor? In the honest case this works well, as MathDial's 1–4% shows. The problem is trust, not capability. A school cannot tell which model or prompt the vendor is running, and Proof-of-Guardrail documents the incentive for providers to silently weaken advertised protections. The adversarial-vendor condition measures what happens when that trust fails. If vendors are assumed honest, this proposal adds little, and we will say so.

Why not just run the same checker at the school, with no enclave? This is the real threat. The school sees outputs anyway, so a local proxy can block leaks just as well. The ablation will likely show identical leak prevention. The enclave earns its place only in the deployment the file describes as common: schools lack the staff to run and secure infrastructure (the "supervision vacuum" in Develop AI Teaching and Learning Resources), a proxy offers parents and regulators nothing they can verify, and a vendor-hosted gate would otherwise expose the answer keys and transcripts to the vendor. We therefore report the enclave's value as auditability and confidentiality, not as better filtering.

Why not just use a learned pedagogical judge inside Proof-of-Guardrail? That judge is the component the adversarial vendor controls the inputs to, and Proof-of-Guardrail itself notes that it can be jailbroken from within. The determinism ablation tests this directly. If the LLM judge's evasion rate turns out to be close to the invariant's, our central claim fails and the taxonomy collapses to "everything is soft."

Proposed Experiments

The main benchmark is MathDial. Its roughly 3,000 problems come with worked solutions, from which we derive step trees and answer keys automatically and then have teachers correct them. We use the MathDial student-simulation setup to drive multi-turn sessions, and we measure leakage by MathDial's turn-level criterion using blinded human annotation. For breadth we add the 25 LeetCode problems with step trees from DBox, where "leaking" means emitting a complete solution for an unvisited step and code equivalence is checked by running tests. Soft properties are scored on AIriskEval-edu, extended with multi-turn traces. No reported result was found on Lacuna for multi-turn pedagogical auditing, so this is new ground and we report our own reference numbers. SciAux tests whether misleading auxiliary context injected by the vendor shifts the soft judge. No reported number was found on Lacuna for SciAux.

The decisive check is the adversarial-vendor table: per property, the evasion rate for invariant versus learned gate under a fixed red-team budget. Systems cost is measured against Proof-of-Guardrail's reported 25–38% latency increase and 18.5x instance cost. Because our gate is a small CPU program plus a classifier rather than a full guardrail model, we expect overhead well below that, and we report it at school-realistic concurrency. Tamper tests on the gate, policy, and report are expected to be detected at the 100% rate Proof-of-Guardrail reports. A negative result there would indicate an implementation bug, not a research finding.

Finally, we test whether teachers can author the policies. Twelve teachers each author step trees for a problem set. We measure false blocks caused by mis-specified keys and the time spent authoring, since a guarantee over a wrong policy is worthless.

The idea fails if paraphrase and multi-turn splitting drive invariant evasion to roughly the learned judge's level. In that case attestation cannot certify pedagogical restraint, and publishing that negative result is itself the correction Proof-of-Guardrail's authors asked for.