Enterprises increasingly train models where their data already lives, inside the relational database, and they face legal requests to erase a person's data under the "right to be forgotten." Deleting the person's rows takes one SQL DELETE. The model trained on those rows still carries their influence. Two facts make this hard. First, a person's data is spread across many linked tables, so one deleted customer can remove thousands of joined transaction records. Second, the fast way to remove a record's influence is to keep running totals and subtract the record's share, and a known result shows that these hidden totals can themselves leak the deleted data. We propose a database-native unlearning method for models trained directly on normalized tables. The database stores only a noised version of the model's summary statistics, in which each person's contribution is bounded. Deleting a person subtracts their exact contribution. The summary left behind then has exactly the same probability distribution as one built from scratch, with fresh noise, on the remaining data, and this holds even when users decide to delete based on what earlier models showed them.
In two sentences: train square-loss models from per-person-bounded, noised sums that the database computes over the joins without ever building the joined table, and keep only the noised sums. When someone invokes their right to be forgotten, compute their cascaded contribution with a small join restricted to their key and subtract it. The result is fast, exact relative to retraining, and gives no extra leakage across repeated releases.
Core Claim
Factorized in-database learning computes models such as ridge regression, factorization machines and PCA from sum-product aggregates over the join without building it. In-Database Factorized Learning does this by pushing sums through joins. AC/DC (Learning Models over Relational Data Using Sparse Tensors and Functional Dependencies) adds sparse tensors and functional dependencies and reports up to 1031× speedups over MADlib, libFM and TensorFlow. Both assume a static database. Cao et al. (Towards Making Systems Forget with Machine Unlearning) showed that a model built from sums can forget a record by subtraction, with 100% completeness. Their records were flat, though, and the exact cached sums are precisely the "secret state" that Forget Unlearning: Towards True Data-Deletion in Machine Learning proves leaks across sequences of releases. The same paper shows that differential privacy for the remaining records is necessary once deletion requests are adaptive.
Our claim is that a single construction answers both papers' concerns at once, for square-loss models over relational schemas. Fix a data-subject relation, such as Users. For subject $u$, let $C_u$ be the sum over all join tuples $t$ that descend from $u$ of the outer product $[x_t; y_t][x_t; y_t]^\top$, where $x_t$ is the tuple's feature vector and $y_t$ its target. The database computes $C_u$ as a group-by-key factorized aggregate. The only state the database persists is
$$S = \sum_{u} \mathrm{clip}_B(C_u) + Z,$$
where $\mathrm{clip}_B$ rescales $C_u$ to Frobenius norm at most $B$ and $Z$ is a symmetric Gaussian noise matrix. $Z$ is drawn once and never stored separately from $S$. The model is solved from $S$ in closed form. Deleting subject $v$ is
$$S' = S - \mathrm{clip}_B(C_v),$$
and $C_v$ is computed by a factorized delta query restricted to $v$'s key. The noise is independent of $v$, so $S'$ is distributed exactly like a fresh noisy build on the remaining subjects. Unlearning is therefore perfect at the level of the internal state, not just the released model. The Gaussian mechanism on bounded per-subject contributions gives differential privacy for the remaining subjects, which is the condition Forget Unlearning requires for adaptive deletion privacy. The cost of a deletion scales with the factorized size of $v$'s cascade, not with the database.
Research Program
The first piece is the per-subject factorized aggregate. AC/DC computes one global aggregate per feature pair. We need the same aggregates grouped by subject key so they can be clipped. That means choosing a variable order with the subject key at the root, extending the sparse-tensor treatment of categorical features to grouped aggregates, and measuring how much this costs relative to ungrouped AC/DC.
The second piece is the deletion delta query. We need the cost of $C_v$ when $v$ fans out through one-to-many and many-to-many joins. We also handle deletions of non-subject rows, for example removing a product. Such a deletion changes $C_u$ for every subject joined to that product, so the delta must be recomputed and re-clipped for each affected subject. We bound the cost in terms of the number of affected subjects.
The third piece is a semantic problem that flat unlearning never meets: derived features. If a training row's feature is "average sales of this store across all customers," deleting one customer changes other customers' features. Per-subject contribution is then no longer local. We will characterize which SQL feature definitions keep $C_u$ local to $u$. For derived aggregates, we will release them as separately noised group statistics, so that dependence on other subjects passes through an already-private quantity.
The fourth piece covers non-square losses. Logistic loss does not factorize, a limitation the file attributes to FAQ-style learning. We replace it with a low-degree polynomial surrogate. The surrogate approximates the loss, but unlearning stays exact relative to retraining on that surrogate. We report the accuracy gap against true logistic regression honestly.
Finally, we close a timing side channel. Machine Unlearning for Random Forests notes that how long an unlearning step takes can reveal facts about the deleted record. Here, delta cost reveals cascade size, so we pad each deletion to a bucketed cost.
We will build the prototype as a DuckDB/Postgres-style extension that keeps $S$ as a catalog object, in the "models as data" spirit of Flock (Cloudy with high chance of DBMS). Flock tracks lineage to detect deletion violations, but it contains no unlearning mechanism; our extension supplies one. Schelter's position paper (Three Challenges in Building Industrial-Scale Recommender Systems) motivates this direction but provides no algorithm.
Baselines
The naive reference is exact retraining inside the database with AC/DC-style factorized learning after every DELETE, with no noise. This is also a strong latency competitor, because factorized aggregate computation is linear in database size for acyclic joins and is already very fast. We run it on the same engine, threads and memory as our method, with the same variable orders and the same ridge penalty. Beating it means lower per-deletion latency, with the crossover point reported as a function of database size and cascade size. We expect to lose on very small databases and say so.
The strongest privacy-matched competitor is noisy retraining from scratch: the same clipped, noised statistic rebuilt over the remaining data after each deletion, with fresh noise each time. It gives the same deletion-privacy guarantee as our method. The comparison therefore isolates exactly one thing, whether subtraction matches it in distribution at a fraction of the cost. We give it identical $B$ and noise scale.
The strongest flat-matrix unlearning method is the Projective Residual Update from Approximate Data Deletion from Machine Learning Models. It is run on the materialized, exported join, with export and join time counted. That paper reports PRU "over 3,000 times faster than exact retraining for single-point deletion" on Yelp at $d = 3000$. One user's cascade can contain many rows, so we run PRU with $k$ equal to the cascade size, where its $O(k^2 d)$ cost bites. Beating PRU means faster end-to-end deletion together with exact rather than approximate equivalence to retraining.
The isolating ablation is our pipeline with exact (noise-free, unclipped) sums: Cao et al.'s summation form lifted to joins. It is the fastest variant, so the comparison isolates what noise and clipping cost in accuracy and what they buy against attacks. We also run in-database SISA with horizontal partitioning by subject key, following Machine Unlearning, which reports 4.63× speedup on Purchase. Partitioning shared tables like Products forces replication, and we count that storage.
Why not just…?
Why not just retrain in the database with AC/DC each time? This is a real threat for latency, and our crossover experiment settles it. It does not settle privacy. A deterministic retrain releases two exact models, before and after the deletion, and their difference reveals the deleted subject's sufficient statistics. Forget Unlearning shows that even exact retraining fails under adaptive requests unless the remaining data are differentially private. The honest comparison is therefore against noisy retraining. There, our only claim is speed with identical distributions.
Why not just subtract exact sums, Cao-style, over the join? It is faster and loses no accuracy. But the exact cached aggregate is a secret state, and its successive releases permit differencing. Our attack experiment measures how often the deleted subject's features can be recovered from consecutive releases under both variants.
Why not just export the join and use PRU or Noisy-GD? Export pays the join explosion that AC/DC avoids. PRU is approximate, and its errors accumulate over deletion streams as its precomputation goes stale. The end-to-end latency and drift experiments test this directly.
Why not just train with differential privacy and never unlearn? Differential privacy bounds a record's influence but does not remove it, and Machine Unlearning treats these as distinct requirements. Our method uses differential privacy only to protect the remaining records and makes the deleted record's influence exactly zero. The accuracy cost of the noise is shared by both approaches, and we measure it.
Proposed Experiments
We use the multi-table Retailer, Favorita and Yelp schemas from AC/DC, with Users/customers as the subject relation where the schema allows. We train ridge regression and degree-2 polynomial regression, plus the polynomial-surrogate logistic model on a binarized Yelp rating.
For per-deletion latency on these schemas we found no reported unlearning result on Lacuna for Retailer or Favorita. The file reports only training speedups for AC/DC there. Yelp has PRU's single-point figure above. We measure per-deletion latency across cascade sizes from single rows to the largest users, under deletion streams of up to 10% of subjects. Throughout each stream we track test RMSE against the non-private, unclipped model, and memory relative to plain AC/DC.
Two decisive checks matter most. First, a distributional test across many seeds, such as a two-sample test on the solved weights, should fail to separate our post-deletion state from noisy retraining. Any detectable difference falsifies the exactness claim. Second, a differencing attack on consecutive releases should recover deleted subjects' feature means with high accuracy under the exact-sum ablation, and near chance under our method at the chosen noise level.
The main risk is utility. Clipping biases models dominated by heavy users, and noise in a $d \times d$ statistic grows with $d$. We will report the privacy–accuracy curve over $B$ and the noise level. If acceptable accuracy requires a noise level that gives only weak privacy, that is a negative result worth publishing, because it bounds what deletion-private in-database learning can achieve. A second experiment injects derived store-level features and checks whether the separately-noised-aggregate treatment keeps the guarantee and the accuracy intact.
References
- In-Database Factorized Learning
- Learning Models over Relational Data Using Sparse Tensors and Functional Dependencies
- Towards Making Systems Forget with Machine Unlearning
- Forget Unlearning: Towards True Data-Deletion in Machine Learning
- Machine Unlearning for Random Forests
- Cloudy with high chance of DBMS: a 10-year prediction for Enterprise-Grade ML
- Three Challenges in Building Industrial-Scale Recommender Systems
- Approximate Data Deletion from Machine Learning Models
- Machine Unlearning