Open models are rarely used as released. A base provider trains a model, a second party tunes it for safety, a third adapts it to a domain such as role-play or medical triage, and a fourth deploys it. When the final model turns out to give harmful answers, nobody can say who caused this, or whether it was anyone's aim rather than an unwanted byproduct of chasing something else. This proposal treats that question as an empirical one. The model's history is a chain of weight updates, each made by a known party toward a known objective. We ask two things of each update: whether it was a necessary part of some set of updates that together produced the harm, and whether the harm was instrumental to that party's objective or a side effect it could have avoided cheaply. We answer both by replaying the chain under controlled counterfactuals. We also make the chain itself carry a tamper-evident, exactly reversible record of its own history. For this we use histogram shifting, a data-hiding technique that writes bits into small, fully undoable changes to the weights.
The two-sentence version is this. Store each party's fine-tuning step as a verifiable, undoable link in the model, then assign blame by re-running the chain without that link. Separately, re-run that party's own training with the harm forbidden: if its task barely suffers, the harm was a side effect it negligently allowed, and if the task collapses, the harm was doing work for its goal.
Core Claim
Responsibility for a harmful behavior in a multi-party fine-tuning chain can be computed from replayed counterfactuals rather than argued about. The necessity part should use path-specific necessity in the sense of Beckers' Counterfactual NESS, not plain leave-one-out, because redundant harmful updates are common in practice. The intended-versus-side-effect part should be measured as the task cost of avoidance, not inferred from an assumed-optimal policy. What is new is the pairing of three operational choices that no cited paper makes together. First, the unit of analysis is a party's weight update, not an agent's action. Second, the intention test is an avoidance-cost measurement that works for noisy, imperfect learners. Third, the custody record needed to trust the counterfactual lives inside the weights and can be removed bit-exactly.
Where This Sits
The work relies on three kinds of prior papers: some supply the theory it operationalizes, some document the gap it fills, and some supply or limit its method.
The causal theory comes from two lines. The Counterfactual NESS Definition of Causation and Moral Responsibility for AI Systems define a cause as a necessary element of an actually sufficient set, evaluated along the path that actually fired. They show this handles preemption cases where simple "but-for" tests fail, and they extend it to a graded degree of responsibility. The Reasons that Agents Act: Intention and Instrumental Goals defines intention by redundancy: an outcome is intended if, once it is guaranteed by other means, the agent loses its reason to act. That paper tests intention on CoinRun and a GPT-4 vignette, but its equivalence to graphical incentives holds only for robustly optimal agents. Beckers stays at the level of structural equations with the control condition assumed. Ward et al. stay single-agent and assume optimality. We keep their definitions but apply them to real stochastic training runs among several parties.
Three papers document the gap. Hazards from Increasingly Accessible Fine-Tuning of Downloadable Foundation Models and Participation in the age of foundation models describe how layered adaptation by hard-to-trace parties obscures who introduced a harmful capability. Towards a Unified Model of Accountability Infrastructures builds causal blame graphs from execution logs. It admits that it cannot guarantee those logs are tamper-proof and that graphs built at that granularity blow up in size.
For combining blame across parties, An Axiomatic Approach to Formalized Responsibility Ascription separates responsibility through risk from responsibility through negligence. It proves that the modified product $1-\prod(1-x_i)$ is the unique aggregator meeting its axioms, and it also proves an impossibility result: no single responsibility function can avoid both "voids" and unavoidable blame. Collective responsibility in multiagent settings formalizes those voids and distributes blame with Shapley and Banzhaf values. Both papers limit what we can promise. We will not claim to escape the impossibility theorem. Instead we report which axiom our scores give up and in which scenarios.
The method for custody comes from Reversible Watermarking in Deep Convolutional Neural Networks for Integrity Authentication. It prunes low-entropy convolutional channels, turns selected decimal digits of their weights into integers, and uses histogram shifting to embed a SHA-256 hash. Histogram shifting here means moving the values between a frequent value and an empty one over by one step, so that the frequent value can then encode bits. The paper reports 0% reconstruction error and under ±0.5% ImageNet accuracy change. Its watermark is deliberately fragile, so any fine-tuning counts as an attack. We change the role of fragility. A party does not fine-tune over an intact watermark. It first extracts the watermark, restoring its predecessor's weights exactly, and verifies them. It then trains and embeds a new link. Fragility therefore detects tampering between links rather than forbidding legitimate adaptation. The side-effect literature supplies a cautionary baseline. Avoiding Side Effects in Complex Environments found that naive distance-from-start penalties could not tell harmless change from destructive interference. That is the analogue of blaming whoever moved the weights most.
Research Program
The first component is an in-band custody chain for transformer checkpoints. Guan et al.'s host selection is specific to convolutional layers. We will replace it with an equivalent for transformers: low-activation-entropy MLP neurons and attention heads, measured on a validation pass. Each party's link holds a party identifier, a hash of the exact predecessor weights, a hash of its update and its training recipe, and the previous link's hash. That is a few hundred bits per link, well within the reported capacity of over 88,000 bits for VGG19.
Chaining works as follows. The next party extracts the watermark and so restores the predecessor's weights bit-exactly. It verifies them, trains, and re-embeds the whole accumulated chain plus its own link into the new weights. A missing or broken chain is then itself evidence. We will measure the accuracy cost of embedding and the reconstruction error. We will also measure which ordinary redistribution operations silently destroy the record, such as dtype casting to bf16, re-sharding, and merging. This is a likely weak point and we will report it plainly.
The second component is counterfactual necessity over updates. With $n$ parties (we use $n \le 5$, so all $2^n$ subsets are feasible), a harm event $H$ means the harmful-response rate on a fixed evaluation set exceeds a threshold $\tau$. We evaluate two kinds of counterfactual chain. The "static" version adds only a subset of the recorded updates to the base weights. The "replayed" version reruns each downstream party's recorded recipe starting from the counterfactual predecessor, because a real downstream party would have trained differently on a different starting model. The custody record makes replay auditable, since the recipe hash tells us which recipe to rerun. Party $k$ is a CNESS cause of $H$ if some subset $S$ of the actual updates that contains $k$ is sufficient for $H$ while $S\setminus\{k\}$ is not. Its graded score follows Beckers' degree of responsibility.
The third component is the avoidance-cost intention test. For party $k$ with task objective $L_k$ and harm measure $h$, we rerun its fine-tuning from its actual predecessor under a constraint that holds harm at the predecessor's level. The constraint uses the Lagrangian cost-constrained optimizer of Safe RLHF: Safe Reinforcement Learning from Human Feedback, via PKU-Alignment/safe-rlhf. We define
$$\iota_k = \frac{L_k(\text{constrained rerun}) - L_k(\text{actual})}{L_k(\text{predecessor}) - L_k(\text{actual})},$$
the fraction of party $k$'s task gain that is lost when the harm is forbidden. When $\iota_k \approx 0$, the harm did no work for the goal. It was a cheaply avoidable side effect, which maps onto negligence in the sense of Hiller et al. When $\iota_k \approx 1$, the harm was instrumental, close to what Ward et al. would call intended.
This is a behavioral version of the redundancy test. It replaces "would an optimal agent still act?" with "how much does this actual learner pay to avoid the harm?", which is well defined for imperfect optimizers. Averaging over training seeds gives error bars on $\iota_k$. Final per-party blame combines the causal score and $\iota_k$, and parties are aggregated with the modified product.
Baselines
The naive reference point is last-toucher blame: the party that produced the deployed checkpoint receives full responsibility. This is roughly what happens when accountability defaults to the deployer. Beating it only requires our scores to track ground truth in chains where the harm was introduced upstream, so it is a floor rather than a real competitor.
The strongest simple competitor is but-for ablation. Each party gets a score equal to the drop in harmful-response rate when its update alone is removed, using the same static and replayed recombination, the same evaluation set, and the same compute. It is strong because it is what most practitioners would compute, and it agrees with CNESS whenever there is exactly one harmful update. It counts as beaten only if CNESS assigns correct nonzero blame in our planted redundancy scenarios, where but-for gives zero to each of two independently sufficient harmful parties, without losing accuracy on single-cause chains.
The strongest existing distributional method is Shapley allocation over parties, following Yazdanpanah et al. It is computed exactly over the same $2^n$ subsets with the same replays, so it adds no compute beyond ours. Shapley splits credit evenly among interchangeable contributors. It therefore cannot tell a party whose harm was instrumental from one that merely allowed it. We beat it if adding $\iota_k$ improves agreement with the planted intended/side-effect labels, reported as rank correlation and classification accuracy, at equal causal accuracy.
A weight-distance baseline blames parties in proportion to the norm of their update. It mirrors the distance penalty that Turner et al. found fails to separate harmless from destructive change, and it tests whether blame is just "who moved the weights most."
Two ablations isolate our key ingredients. The first uses static rather than replayed counterfactuals, which tests whether retraining downstream matters. The second replaces $\iota_k$ with Ward-style intention inferred from a single greedy policy comparison without the constrained rerun, which tests whether measuring avoidance cost is what handles imperfect learners.
For intended/side-effect attribution and for fine-tuning-chain blame, no reported result was found on Lacuna. These baselines will therefore be run fresh with matched seeds and budgets. For the constrained optimizer we reuse, the reference number is from Safe RLHF: Beaver-v3 reduced the harmful-response probability from 53.08% (Alpaca base) to 2.45%. For custody, the reference from Guan et al. is 0% reconstruction error with under ±0.5% ImageNet accuracy change.
Why not just…?
Why not just attach signed metadata or store the diffs in a registry instead of embedding anything in the weights? For the blame computation itself, this would work equally well, and we say so. Components two and three consume the updates and recipes regardless of where they are stored. The in-band chain only matters where sidecars get stripped as models are re-hosted and re-tuned, which Chan et al. and Suresh et al. describe as the norm for downloadable models. Our custody experiment tests this directly. We apply realistic redistribution operations, including re-uploading, format conversion, dtype casting and merging. We then count how often sidecar metadata survives versus how often the in-band chain survives or at least fails in a detectable way. If casting to bf16 destroys the embedding as often as re-hosting drops sidecars, the histogram-shifting component loses its case, and we will report the attribution method as standalone.
Why not just use leave-one-out "but-for" ablation? Because harmful updates are often redundant. If both a domain tuner and a later instruction tuner would each have broken safety alone, but-for blame assigns zero to each. Beckers' papers show this failure in the backup-assassin and rock-throwing preemption cases. We plant exactly these structures, early and late preemption translated into update chains, so the gap is measured rather than asserted.
Why not just compute Shapley values and stop? With five parties, Shapley is cheap and handles redundancy by splitting credit, so it is a real threat on the causal side. What it cannot do is separate a party that harmed safety to gain task performance from one that harmed it for nothing. That distinction bears on negligence versus risk in Hiller et al.'s sense. If $\iota_k$ adds no predictive value over Shapley on the planted intended/side-effect labels, the intention component fails. We would then report Shapley over replayed chains as the recommended method.
Why not just penalize harm during every party's fine-tuning and skip attribution? Constrained training in the style of Safe RLHF is the forward-looking fix, and we use it inside the intention test. But the parties who matter most for accountability, the untraceable fine-tuners in Chan et al., will not adopt it voluntarily. Backward-looking attribution is what gives them a reason to.
Proposed Experiments
The ground-truth testbed is a set of constructed chains of three to five parties, starting from open base models of modest size. Base and safety stages use PKU-SafeRLHF preference data. Harmfulness is measured as the harmful-response rate under that dataset's cost model across its 14 harm categories. Domain stages include role-play fine-tuning via yulinlp/SaRFT, which is documented to erode safety as a byproduct.
We plant three kinds of harm. The first is instrumental: a party whose task reward literally pays for compliance with harmful requests. The second is a side effect: benign domain data that erodes refusals. The third is redundant: two parties each independently sufficient. The planted structure is the ground truth for necessity and for the intended/side-effect label.
As a sanity check at small scale, we reproduce the side-effect/instrumental distinction in SafeLife. There, AUP's reported numbers from Turner et al. (98% of PPO reward with half the side effects in append-still-easy, and 111% of reward with 39% of side effects in append-spawn) give known points where side effects were cheaply avoidable. We check that $\iota$ computed for plain PPO is correspondingly low.
The decisive checks are as follows. CNESS must beat but-for ablation on redundant chains while matching it on single-cause chains. $\iota_k$ must separate planted instrumental harm from planted side effects with non-overlapping seed intervals, and it must add value over Shapley. Replayed counterfactuals must change the attributions relative to static delta arithmetic in at least the redundant scenarios; otherwise the cheaper static version suffices. Finally, the custody chain must show near-zero reconstruction error and negligible benchmark change on transformer checkpoints, and it must survive, or fail detectably, under redistribution more often than sidecar metadata. A clean negative result on this last check is acceptable and would reduce the contribution to the attribution method alone.
References
- The Counterfactual NESS Definition of Causation
- Moral Responsibility for AI Systems
- The Reasons that Agents Act: Intention and Instrumental Goals
- Hazards from Increasingly Accessible Fine-Tuning of Downloadable Foundation Models
- Participation in the age of foundation models
- Towards a Unified Model of Accountability Infrastructures
- An Axiomatic Approach to Formalized Responsibility Ascription
- Collective responsibility in multiagent settings
- Reversible Watermarking in Deep Convolutional Neural Networks for Integrity Authentication
- Avoiding Side Effects in Complex Environments
- Safe RLHF: Safe Reinforcement Learning from Human Feedback