Privacy law gives people the right to have their data deleted, and for a trained neural network that means the model must behave as if it had never seen that data. Current deletion ("machine unlearning") methods treat every training example the same, whether the model absorbed it in its first epochs or long after training had settled. Deep networks, however, have critical periods. Early in training they commit to low-level features that later clean data cannot undo, much as early growth faltering in children persists along parallel trajectories rather than catching up (Causes and consequences of child growth faltering in low-resource settings; Early-childhood linear growth faltering in low- and middle-income countries). This note proposes that the same path-dependence governs forgetting. Data present during a network's critical window gets built into the early feature extractor and resists cheap erasure. Data added after training has consolidated sits mostly in the late layers and can be removed cheaply.
We test this in concept bottleneck models: networks that first predict human-readable attributes, such as "wing color" or "joint space narrowing", and then predict the label from those attributes alone. These models matter here for two reasons. They are the interpretable architecture for which fast closed-form deletion already exists (Controllable Concept Bottleneck Models). Their concept labels also give us a direct instrument for checking whether a "deleted" concept can still be read out of the feature extractor. The deliverable is a measured map of deletion difficulty against acquisition time, plus a simple deletion policy that uses each request's training history to decide how much of the network it must touch.
Core Claim
Residual, recoverable information about deleted data depends strongly on when that data entered training relative to the network's critical window, holding total exposure fixed. By residual information we mean what an attacker can extract from intermediate activations, as opposed to what the output reveals. Two consequences follow. First, deletion methods that edit only the bottleneck and late layers, including closed-form influence updates, will pass output-level privacy tests on early-entered data while leaving the concept decodable from the backbone. They will conceal it rather than erase it. Second, a deletion policy that conditions on entry time should reach retraining-level erasure at a fraction of the compute of always rewinding or always retraining.
The claim is falsifiable. If decodability after unlearning is flat across entry times, or if a plain per-example memorization score predicts difficulty just as well, the developmental story adds nothing and the right answer is existing post-hoc unlearning with better audits.
Positioning
Three papers bracket the idea, and none tests it. Critical Learning Periods in Deep Networks motivates it. That paper shows that blur during roughly the first 40–60 CIFAR-10 epochs permanently raises test error from about 6% to over 18%. Sensitivity peaks near epoch 30, tracked by the trace of the Fisher information, a measure of how strongly the loss depends on each weight. The effect is concentrated in low-level features. However, that paper studies only acquisition, never removal, and only in black-box networks.
Controllable Concept Bottleneck Models supplies the method we stress-test. It uses two-stage influence functions, a second-order estimate of how weights would change without given data, to delete samples or concepts from a bottleneck model without retraining. Its audit uses output membership inference, an attack that guesses from model outputs whether an example was in training. It also assumes local convexity at a converged point and ignores training history entirely.
Unlearning Information Bottleneck removes systematic patterns through a variational bottleneck. Its latent space is not human-readable, and it too is timing-blind.
Two further lines constrain the design. Unlearning or Concealment? found that erased concepts in diffusion models reappear when generation starts from the base model's partial trajectory. Does Machine Unlearning Truly Remove Knowledge? found that activation perturbation exposes knowledge in language models after unlearning. Together they mean output metrics cannot be our verdict. On the Plasticity Collapse in Continual Machine Unlearning shows that sequential deletion requests either stop working or bring back old data. That warns us that any policy must be tested on request streams, not single deletions.
Research Program
We first build a controlled timing testbed. Training data is split into cohorts that enter at different points, which mimics a deployed model that keeps absorbing new users. One cohort is present from epoch 0. One enters at the measured Fisher peak. One enters only after the Fisher trace has fallen, which is the consolidation phase. We match the number of gradient steps each cohort receives, so that timing is not confounded with exposure. We also match cohort content, either by random user-like groups or by groups defined by a concept such as a CUB attribute or a CelebA attribute.
During training we keep a lightweight "developmental ledger". For each cohort it records the entry epoch, the network-wide and layer-wise Fisher trace at entry (estimated from squared minibatch gradients, as in the critical-periods work), and checkpoints at cohort entry points.
Second, we measure deletion difficulty against entry time. Each cohort, or each concept, is deleted with a fixed set of methods. For each method we measure three things. The first is output membership-inference AUC. The second is residual decodability: a linear probe trained on held-out data to read the deleted concept, or cohort membership, from each backbone block, together with an activation-perturbation probe in the style of Does Machine Unlearning Truly Remove Knowledge?. The third is relearning speed, meaning how many steps a fine-tune needs to recover accuracy on the deleted data. Every measurement is compared against a model retrained from scratch without that cohort. The concept annotations matter here. In a bottleneck model we can ask whether "wing color" is still linearly present in block 2 after it has been pruned from the bottleneck. That is a concept-level erasure audit that black boxes do not allow.
Third, we build the timing-aware deletion policy. For a request, the ledger says whether the data entered during or after the critical window, and which layers gained the most Fisher information at entry. Post-consolidation requests get the cheap closed-form bottleneck edit. Critical-window requests get one of two treatments. The first is projected-gradient unlearning restricted to the flagged early layers, using the projection idea of Learn to Unlearn for Deep Neural Networks, which confines updates to directions orthogonal to the retained data's activations. The second, used if that fails the probe audit, is a rewind to the nearest pre-entry checkpoint followed by replay without the cohort. The policy is deliberately simple. The scientific contribution is the measured timing dependence and whether conditioning on it pays.
Fourth, we run request streams. Policies face 10 to 50 sequential deletions with mixed entry times, and we track the forward and backward failure modes defined in On the Plasticity Collapse in Continual Machine Unlearning.
Baselines
The naive reference is concept zeroing. We delete the concept's weights in the label predictor, or the cohort's influence only at the output layer, in the spirit of Boundary Unlearning. This is what a practitioner would try first in a bottleneck model. It should pass output tests and fail probes for all cohorts. Retrain-from-scratch without the cohort is the gold-standard reference that defines "erased". It is also the compute ceiling. Fine-tuning on retained data and NegGrad+ are the cheap gradient baselines. They are run with the same epoch budget as our policy.
The strongest existing method on this question is the closed-form influence editing of Controllable Concept Bottleneck Models. It is the only method built for deletion in bottleneck models. On OAI it reports a 2.36-minute update versus nearly 300 minutes for retraining, with F1 within 0.002 of retraining, membership-inference indistinguishability, and stability over 10 editing rounds. We run it with the same ResNet-18 backbone, the same EK-FAC curvature approximation it uses, and the same cohorts. Projected-gradient unlearning is run on the same models as the strongest black-box-style method. It reports a CIFAR-10 membership-inference AUC of 0.5340 at 88 s versus 1021 s for retraining. Absolute concept-level probe results for either method: no reported result found on Lacuna. Establishing them is part of the contribution.
Two ablations isolate the key ingredient. The first replaces the entry-time signal with a timing-free difficulty score: per-example training loss or a memorization estimate computed at the end of training. This tests whether timing carries information beyond "how memorized". The second replaces the timing signal with random routing at matched compute.
Our policy beats the strongest baseline if it meets three conditions on critical-window cohorts. Backbone probe accuracy must fall to within the retrain model's confidence interval where the influence edit does not. Retained accuracy must stay within one point of retraining. Total compute must be well below always-rewind. For post-consolidation cohorts the policy should simply match the influence edit, since it uses it.
Why not just…?
Why not just zero the concept in the bottleneck, or use the existing influence edit? Both are output-side or locally linearized operations. The concealment results in diffusion and language models (Unlearning or Concealment?; Does Machine Unlearning Truly Remove Knowledge?) suggest that information can survive in intermediate layers. The influence edit also assumes a local quadratic loss, which is least plausible for data that shaped the early basin. This is an empirical question, and the timing-by-probe experiment settles it. If the influence edit leaves no decodable residue even for epoch-0 cohorts, our proposal reduces to a validation of that method, and we will report it as such.
Why not just rewind to a checkpoint and retrain for every request? This is a real threat, because it gives near-exact erasure. Its cost scales with the time since the cohort entered. For data present from epoch 0 it is full retraining, at roughly 300 minutes per request on OAI per Controllable Concept Bottleneck Models. For late data it is wasteful if a two-minute edit achieves the same probe-level erasure. Our compute-versus-erasure curves across mixed request streams show whether selective use of rewinding beats uniform use. If late-cohort influence edits fail probes too, uniform rewinding wins and we will say so.
Why not just use a memorization score instead of training timing? Hard-to-forget data might simply be data the model fit tightly, regardless of when it entered. The first ablation tests this directly with content-matched and exposure-matched cohorts. Timing earns its place only if it predicts residual decodability after conditioning on the memorization score.
Why not just use a black-box network with projected-gradient unlearning? It may erase as well. What it cannot provide is a concept-level audit or test-time correction of concepts. The bottleneck costs some accuracy, with CUB error 19.9% versus 17.5% for a black box in Concept Bottleneck Models. We run both architectures on the same cohorts. We report whether the bottleneck changes timing sensitivity, for instance by localizing late data more cleanly, and what that accuracy price buys in deletion.
Proposed Experiments
We start on CIFAR-10 with ResNet-18, both as a black box and with a bottleneck over attribute-like labels. This reproduces the Fisher peak near epoch 30 before anything else. If we cannot reproduce it, the timing manipulation is undefined. We then move to CUB-200-2011 with its 312 attributes and to OAI with its clinical concepts, the two datasets shared by the bottleneck and influence-editing papers. CelebA concept deletion serves as the sensitive-attribute case. CUB + Places tests whether spurious background associations, which bottleneck models resist (48–49% versus 62.7% error under shift in Concept Bottleneck Models), are harder to erase when present from epoch 0.
The decisive check is a single plot per dataset. It shows post-unlearning backbone probe accuracy, minus the retrain model's, against cohort entry epoch, for each method, with exposure held fixed. Our hypothesis predicts a gap that is large for epoch-0 and Fisher-peak cohorts and near zero for post-consolidation cohorts under bottleneck-only and influence edits. It also predicts that the gap persists after controlling for the memorization score.
A second check is the request-stream experiment. We test whether routing early-cohort requests away from repeated second-order edits avoids the forward-failure degradation of 33.2% and the reappearance of deleted data reported by On the Plasticity Collapse in Continual Machine Unlearning. A flat timing curve, or a memorization score that explains it fully, would falsify the proposal. Either outcome would still give the field a probe-level audit of bottleneck-model deletion that does not currently exist.
References
- Causes and consequences of child growth faltering in low-resource settings
- Early-childhood linear growth faltering in low- and middle-income countries
- Controllable Concept Bottleneck Models
- Critical Learning Periods in Deep Networks
- Unlearning Information Bottleneck: Machine Unlearning of Systematic Patterns and Biases
- Unlearning or Concealment? A Critical Analysis and Evaluation Metrics for Unlearning in Diffusion Models
- Does Machine Unlearning Truly Remove Knowledge?
- On the Plasticity Collapse in Continual Machine Unlearning
- Learn to Unlearn for Deep Neural Networks: Minimizing Unlearning Interference with Gradient Projection
- Boundary Unlearning: Rapid Forgetting of Deep Networks via Shifting the Decision Boundary
- Concept Bottleneck Models