Virtual conferences let people take part who could never afford a flight or a visa. VLDB 2020 drew over 5,000 registrants once it went online, and student symposia now mix in-person and remote cohorts. These events also produce data: recorded Q&A, paper-page chat, which posters each attendee opened. Organizers increasingly feed that data into helper models, such as paper recommenders, semantic poster maps, and question-answering assistants. When an attendee later asks to be erased under the "right to be forgotten," deleting their rows does not remove what the trained model absorbed from them. Machine unlearning is the set of techniques that edit a trained model so it behaves as if it had never seen certain data. It is the standard answer, but it is usually judged by averages over the whole remaining population.
This note argues that averages hide the case that matters most here. Accessibility measures bring in participants who are few, sparse in their activity, and clustered: remote attendees, people in distant time zones, early-career researchers, and quiet "lurkers." We hypothesize that approximate unlearning damages these groups more than it damages the majority. One reason is that the forget requests often come from them. A second reason is that the "protected" directions some methods preserve are chosen by variance, and majority attendees dominate that variance. If this is true, a privacy mechanism built into inclusive infrastructure would quietly degrade service for the people that infrastructure was built to include. We propose to measure this effect, explain it mechanistically, and test a simple geometric fix.
Core Claim
Approximate unlearning in conference-scale systems produces collateral utility loss and residual leakage that concentrate on minority participant groups. For projection-based methods there is a specific cause. Projected-Gradient Unlearning (Learn to Unlearn for Deep Neural Networks) runs a singular value decomposition on retain-set layer representations. It keeps the top directions as a "core gradient space" that unlearning steps must avoid, and it pushes the forgetting update into the leftover "residual" directions. Those top directions are fit to whichever group supplies most of the variance. Low-variance directions that encode rare groups therefore land in the residual space, which is exactly where the unlearning update is allowed to act. We predict the gap closes when the core space is computed from group-balanced statistics:
$$M = \sum_{g} \frac{1}{n_g} R_g^{\top} R_g ,$$
Here $R_g$ is the matrix of layer inputs from retain-set members of participant group $g$, and $n_g$ is that group's size. The core space is the span of the top eigenvectors of $M$. These are replaced into PGU unchanged. The claim is falsifiable at three points: the disparity may not exist, the mechanism may not explain it, or the fix may fail to reduce it without increasing leakage.
Research Program
The literature motivates this question without answering it. Benchmarks for the right to be forgotten in interactive systems test commercial recommenders. ERASE evaluates seven unlearning methods across recommender families on e-commerce and media data and finds divergence under repeated requests. E2URec unlearns an LLM recommender through LoRA adapters, small trainable add-on weight matrices, using two teacher models. Neither paper reports results split by user subgroup. TOFU builds 200 fictitious author biographies to test entity-level forgetting in LLMs and shows that standard baselines either fail to forget or wreck utility. It treats authors as interchangeable. On the conference side, MiniConf is the open-source platform behind ICLR 2020 and ACL. It computes document embeddings for a 2D poster map and embeds chat on paper pages, but it has no deletion path. Ding et al. argue that real participation requires accepting "lurking" and peripheral contribution, and lurkers are precisely the users who leave sparse behavioral traces. Hybrid events such as ISSABC (101 in-person, 79 online) show that remote attendees form a real but smaller cohort. Together these papers supply the regime, minority and sparse participants, and the gap, which is the absence of any subgroup-disaggregated unlearning evaluation.
We will build three things. The first is a synthetic conference benchmark that extends TOFU. TOFU's fictitious authors become presenters. We generate fictitious attendees with attributes: attendance mode, region, time zone, and activity level. The attendee population is imbalanced on purpose, with mixes from 90/10 to 60/40 that track the hybrid cohorts reported above. We then generate their Q&A questions, chat messages, and poster-visit logs. A conference assistant, a TOFU-scale fine-tuned LLM, answers questions about talks and discussion. A MiniConf-style recommender is trained on visit logs. Forget requests arrive at the level of a whole attendee and in sequence, because ERASE shows that sequential requests are where approximate methods break. Synthetic data is necessary because no real conference interaction log with participant attributes can be released, and erasing real people to study erasure would undercut the point.
The second is a disaggregated evaluation. For each forget request we measure forget quality using TOFU's statistical test and membership-inference leakage. We report utility separately for the forgotten person's own group and for the other groups.
The third is the mechanistic analysis plus the group-balanced core-space fix. For the analysis, we measure what fraction of each group's representation energy falls in PGU's residual space. We then check whether that fraction predicts the group's collateral loss.
Baselines
The naive reference point is row deletion alone. The attendee's data is removed from the store and the deployed model is left untouched. It sets the ceiling on leakage, and every method must clearly improve on its membership-inference score.
The gold reference is retraining from scratch on the retained data with an identical training recipe and seed protocol. Its subgroup utility profile defines "no disparity introduced by unlearning." Any method's disparity is measured relative to retraining, not in absolute terms. This matters because a model trained on imbalanced data already serves minorities worse, and we want to isolate the damage that unlearning adds on top.
The strongest existing projection method is PGU, run with its published dual loss of reverse cross-entropy plus entropy maximization, and with its core-space energy threshold tuned on a held-out request set. On CIFAR-10 it reports a membership-inference ROC AUC of 0.5340 against a 0.50 random-guess floor, retain error within 0.01% of retraining, and an 11.6x speedup (88s versus 1021s) (Learn to Unlearn). Because PGU constrains its updates more than the other methods, it is the method most likely to look "safe" on averages.
For the LLM assistant, the strongest competitors are TOFU's Gradient Difference and KL Minimization baselines. TOFU reports that at a 1% forget set these often fail to reach significant forget quality (p > 0.05), and that at 5–10% utility collapses. In-context unlearning (Pawelczyk et al.) beat gradient ascent in 14 of 16 cases under LiRA-Forget and is included as the weight-free option.
For the recommender, the strongest baseline is SCIF, which ERASE identifies as the most consistent method. There is no reported number on Lacuna for SCIF's subgroup-level utility, and none for any method on conference data. LoRA-only unlearning in the style of E2URec is the parameter-efficient competitor; it reports a 10x speedup over retraining on MovieLens-1M while updating 0.7% of a T5-base model.
All methods get the same base checkpoints, the same request sequences, and the same hyperparameter search budget, measured in GPU-hours on the held-out request split.
Two ablations isolate the key ingredient. The first is PGU with balanced retain sampling but an unweighted SVD. It tests whether reweighting the data alone does the work. The second keeps the group-balanced core space but randomly permutes the group labels. It tests whether gains come from group structure or simply from a larger, more diffuse core space.
Beating the field means one thing. The gap between minority and majority collateral loss, measured relative to retraining, must shrink by a margin larger than the seed-to-seed variance, while membership-inference AUC on the forgotten attendee stays no worse than PGU's and average retain utility stays within retraining's noise band.
Why not just…?
Why not just retrain from scratch? At conference scale this is a real threat. A recommender over a few thousand papers and attendees may retrain in minutes, and in that case retraining is the right answer and we will say so. The reason to study approximate methods is the LLM assistant and the live-event window. ERASE argues that interactive systems need unlearning on the order of 1,000x faster than retraining, and requests during a multi-day event arrive in sequence. We will report the retraining wall-clock time at each scale. Our disparity findings apply only above the scale where retraining stops fitting the request window.
Why not just build the assistant on retrieval, so deleting a record is exact? This could make parametric unlearning unnecessary for the Q&A assistant. We test it directly with a retrieval-only assistant over the same transcripts, comparing per-group answer quality against the fine-tuned assistant. If retrieval matches fine-tuning for minority attendees, the recommendation becomes architectural: keep personal data out of the weights. The study then reduces to the recommender and embedding components, which learn from behavior and cannot be made purely retrieval-based.
Why not just oversample minority attendees in the retain set during unlearning? This is the cheapest fix and the first ablation. If our mechanism is correct, oversampling helps loss-based methods such as Gradient Difference. It helps PGU less, because PGU's core space comes from representation variance and not from the loss. If oversampling alone closes the gap for PGU, the geometric story is wrong, and we will report it that way.
Why not just report averages more carefully, for instance with confidence intervals? Averages over an imbalanced population can stay flat while one group degrades sharply. The first decisive check below asks whether disaggregation reveals anything averages miss. If it does not, the measurement contribution fails and that is a negative result worth publishing.
Proposed Experiments
The first decisive check is whether the disparity exists. On the synthetic conference benchmark, with imbalanced attendee mixes and 1–50 sequential attendee-level requests, we compute each method's collateral utility loss relative to retraining for the forgotten attendee's group and for the other groups. The hypothesis fails if the minority-minus-majority gap for PGU, Gradient Difference, SCIF, and LoRA unlearning is within seed variance across all imbalance ratios. We replicate on the ERASE recommender datasets, using user activity tertiles as a proxy for peripheral participants, so the result does not depend on our generator.
The second check is mechanistic. For PGU, we regress each group's collateral loss on the fraction of that group's retain-representation energy that falls outside the core space. We repeat this across layers and energy thresholds. A strong positive relation supports the cause we propose. No relation means the disparity, if present, comes from somewhere else, and the fix has no reason to work.
The third check is the fix itself. Group-balanced PGU is compared with standard PGU and both ablations. We report the per-group gap, membership-inference AUC on forgotten attendees, average utility, and runtime. As a sanity check that the change does not break PGU's original setting, we also run it on CIFAR-100 with deliberately imbalanced subclasses inside superclasses, where we expect it to match PGU's reported behavior on the balanced data.
The fourth check is the retrieval-only comparison described above. It decides whether the assistant result is a statement about unlearning or about architecture.
Together these checks tell an organizer something concrete. Either honoring erasure requests is neutral for the remote and peripheral attendees their platform exists to serve, or it is not, and in that case we will know which design choice causes the harm.
References
- Learn to Unlearn for Deep Neural Networks: Minimizing Unlearning Interference with Gradient Projection
- ERASE -- A Real-World Aligned Benchmark for Unlearning in Recommender Systems
- Towards efficient and effective unlearning of large language models for recommendation
- TOFU: A Task of Fictitious Unlearning for LLMs
- MiniConf - A Virtual Conference Framework
- Towards Openness Beyond Open Access: User Journeys through 3 Open AI Collaboratives
- Beyond barriers, towards diversity: how hybrid student conferences can drive accessibility
- In-Context Unlearning: Language Models as Few-Shot Unlearners