Systems that learn what people want from logged interactions now try to track preferences that change over time. The standard trick is to discount old feedback, as non-stationary preference optimization does. These methods assume the log contains everything people said. It does not. Moderators, platform filters and users themselves delete content, and they delete the angriest, most negative and most extreme material first and fastest. On Wikipedia talk pages, 82% of severely toxic comments disappear within a day. Telegram channel admins delete 10–50% of what they post.
Deletion policies also change over time. A preference learner that only sees what survived can therefore mistake a change in what gets deleted for a change in what people prefer. This note proposes to measure that confusion and to correct it. We weight each logged preference by the inverse of its estimated chance of surviving to collection, alongside the usual time discount. The survival model is fit on small "dual-collected" slices, where both the deleted and the surviving content are known.
The core claim fits in two sentences. Time-discounted preference learning on moderated logs is biased in a specific, testable way: it reports drift when only moderation changed, and it underweights dissatisfaction that was deleted. An estimate of each item's survival probability, built from its content and the timing of its deletion risk, removes most of that bias, provided a modest calibration slice of uncensored data exists.
Why these directions meet here
The four directions play unequal roles. Moderation-induced survivorship bias, meaning the distortion that comes from studying only what survived deletion, is the core mechanism. Temporal preference drift is the application regime. Explanation scarcity explains why the bias matters so much. The cross-modal gap is a design constraint on one component, and we test it rather than assume it.
The motivating evidence is strong. WikiConv reconstructs full Wikipedia talk-page histories from revision diffs, with 98% accuracy in recovering structure. It finds that nearly 40% of conversational activity is invisible in final snapshots, and it documents the 33% and 82% within-24-hour deletion rates for toxic and severely toxic comments. A Telegram Dataset of Propaganda and its Moderation compares real-time streams against later exports and shows that admin deletion is selective with respect to narrative rather than random. That kind of missingness defeats ordinary imputation.
Alexandros Efstratiou warns that apparent community depolarization can be an artifact of banning toxic actors. This is precisely a case of moderation masquerading as behavioural drift. The 14-year cohort study in Tracking patterns in toxicity and antisocial behavior over user lifetimes reports a reversal around 2013: users became less toxic over their lifetimes before it and more toxic after it. The authors themselves concede they cannot separate platform intervention from societal change. These papers motivate the proposal. None of them trains a preference model.
The method we extend is Non-Stationary Direct Preference Optimization (NS-DPO). Direct preference optimization fits a language model directly on pairs of preferred and rejected responses, without a separate reward model. NS-DPO adds an exponential discount $\gamma^{T-t_i-1}$ by age to each pair, under a time-varying Bradley–Terry model of choices. It assumes the dataset is a complete record that has merely aged. We keep its discount and drop that completeness assumption.
Explanation scarcity sharpens the stakes. Explicit positive feedback comes from only 1–3% of users, so DRIFT learns from dissatisfaction instead: corrections and complaints serve as the rejected side of a pair. Those complaints are often the only place a user says why something was wrong. Similarly, CIPHER turns user edits into natural-language descriptions of preferences. Both rely on implicit, explanation-bearing negative signals, and those are exactly the signals deletion removes first. Our proposal does not generate rationales. Instead, it argues that the scarce explanation data is scarce partly because of moderation, and it corrects for that loss.
Core Claim and Estimator
Let each logged preference pair $i$ carry a timestamp $t_i$, out of $T$ periods. Let $\hat\pi_i$ be the estimated probability that the pair survived to collection, meaning both responses and the feedback signal were still present when the log was crawled. Let $\ell_i(\theta)$ be the standard pairwise preference loss for model parameters $\theta$. We train with
$$ \mathcal{L}(\theta) = \sum_i \frac{\gamma^{T-t_i-1}}{\max(\hat\pi_i, c)}\,\ell_i(\theta), $$
where $c$ is a clipping floor that bounds the variance of the weights. Setting $\hat\pi_i=1$ recovers NS-DPO. Setting $\gamma=1$ recovers plain inverse-propensity DPO.
Survival is modeled as a hazard over the item's age. The probability that item $i$ survives to its crawl time $a_i$ after posting is $\hat\pi_i = \exp(-\int_0^{a_i} h(s \mid x_i, z_i)\,ds)$. Here $h$ is the deletion hazard, meaning the instantaneous deletion rate at age $s$. It depends on the text content $x_i$ and on numeric timing telemetry $z_i$: reply velocity in the thread, author history, and the platform's moderation-regime period.
The hazard form is what lets moderation that changes over time enter the estimator. When a platform tightens its filters, the hazard rises for the affected content and that era's survivors get larger weights. The hazard is fit on calibration slices where deletions are observed: WikiConv's full action history and the Telegram dual stream.
Identifiability rests on one stated assumption. Given $x_i$ and $z_i$, deletion must be independent of the preference label, and the calibration slice must transfer to the main log. We test transfer directly rather than assume it.
Research Program
The first deliverable is a semi-synthetic benchmark with known ground truth. We start from the non-stationary UltraFeedback and preference-flip settings used for NS-DPO. We apply deletion masks whose hazard curves are fit to WikiConv's empirical deletion-by-toxicity-and-age profile and to Telegram's 10–50% selective deletion.
Crossing preference behaviour with moderation behaviour gives three regimes:
- Drift only: true preference drift under constant moderation.
- Moderation change only: constant preferences under a mid-stream moderation tightening.
- Both: drift and a moderation change together.
Every method trains on the survivors and is evaluated against the uncensored preference distribution. The second regime is the diagnostic one. Any method that reports drift there is reading moderation as preference.
The second deliverable is a dissatisfaction variant built on the DRIFT setup. Dissatisfaction turns are deleted at elevated, time-varying rates, which mimics users deleting angry messages and moderators removing hostile ones. This tests whether survival weighting restores the value of the scarce negative signal.
The third deliverable is a real-data measurement study on WikiConv. We compute user-lifetime toxicity slopes, the quantity behind the post-2013 reversal, three ways: on snapshot-only data, on full history, and on snapshot data reweighted by $\hat\pi$. If reweighting moves the snapshot estimate most of the way to the full-history estimate, the estimator works on real moderation and not only on simulated deletion masks. This partially addresses the causal question the cohort study left open, for Wikipedia.
Transfer is tested by fitting the hazard on English WikiConv and applying it to Chinese WikiConv, and by fitting on early periods and applying to later ones. We also vary the calibration slice from 1% to 20% of items to find how much dual-collected data the correction needs.
The cross-modal question is confined to the hazard model's handling of $z_i$. We compare four ways of reading the timing telemetry. The first serializes it as text in an LLM prompt. The second uses a small tabular or recurrent encoder concatenated with a frozen text embedding. The third uses a patch-projection encoder that maps short telemetry windows into a frozen language model's input embedding space, in the style of Time-LLM. The fourth adds cross-modal distribution alignment between the numeric and text branches, as in CALF. FoundTS finds that language-model backbones often lose to small dedicated numeric models because text priors interfere with numbers. Our prior is therefore that the small encoder will suffice, and we will report it if so.
Baselines
The naive reference point is standard DPO trained on the surviving pairs, which ignores both time and deletion. We expect it to fail on drift and to be misled by moderation. The upper reference is an oracle that runs NS-DPO on the full uncensored data. It shows how much of the gap the correction closes.
The strongest existing competitor is NS-DPO run on survivors. It is the only method in the file that targets drift directly, and it has published wins. It reaches a length-controlled win rate of 8.93 against 7.29 for DPO on UltraFeedback-LM. Under preference flips it keeps reward accuracy above roughly 60–70%, while DPO falls to about 50–55% (Right Now, Wrong Then).
To make the comparison fair, NS-DPO gets the same backbone, data, steps and compute. It also gets the same tuning budget, spent on its discount $\gamma$, which is the knob a skeptic would say should absorb attrition. We count it as beaten only if our estimator does both of the following:
- Closes a majority of the reward-accuracy and win-rate gap between survivor-NS-DPO and the oracle in the moderation-change regimes.
- Shows no loss in the drift-only regime.
For the dissatisfaction variant, the competitor is DRIFT trained on survivors, under the same budget. DRIFT reports WildBench task-score gains of +6.23% at 7B and +7.61% at 14B on its own uncensored real-world data. There is no reported result found on Lacuna for DRIFT or NS-DPO under deletion-censored logs, so the censored numbers will be produced here.
A sliding-window DPO baseline represents the simplest drift heuristic, and its window size gets the same tuning budget. A uniform-rate correction reweights every item by the platform's overall deletion rate. It tests whether item-level hazards matter at all.
Two ablations isolate the key ingredients. The first keeps the time discount but drops survival weights, which is NS-DPO itself. The second fits survival weights from text content only, with no age or telemetry, so it ignores when deletion risk accrues. If the text-only ablation matches the full model, the timing and hazard component is unnecessary and we will say so.
Why not just…?
Why not just tune NS-DPO's discount factor harder? A discount reweights by age alone. Suppose moderation tightens in later periods. Then recent survivors are missing negatives that older survivors still contain, and no single $\gamma$ can tell that apart from a genuine preference shift. Shrinking $\gamma$ trusts the distorted recent data more. Growing it ignores real drift. The NS-DPO paper itself notes that $\gamma$ depends on knowing the variation budget in advance and cannot distinguish a good old point from an obsolete one. The moderation-change-only regime settles this. If a tuned $\gamma$ achieves near-oracle accuracy there, our proposal is unnecessary.
Why not just collect complete data with real-time streaming? Real-time collection is the gold standard, as the Telegram dual-collection and WikiConv reconstruction show. But it cannot be applied retroactively to existing preference logs. It is often infeasible for private product logs, where deleted conversations are purged by policy. Even WikiConv-style reconstruction failed on archived and relocated threads in about 24% of its error cases. This alternative is complementary rather than competing: the proposal uses a small real-time slice for calibration. The calibration-size sweep measures how little complete data is enough. If it turns out to need most of the log, the alternative wins.
Why not just write the telemetry into the prompt as text, or skip language models for it? Serialized numbers split into sub-word tokens and lose their scale. CALF reports a 7.05% MSE reduction over PatchTST from explicit cross-modal alignment, and the time-series literature summarized in the file documents distribution divergence when numbers are treated as text. On the other side, FoundTS shows small numeric models often beat language-model backbones outright. That second point is a real threat to any language-model-based telemetry encoder. We therefore treat the encoder as an empirical question in the four-way comparison and default to the small encoder unless an alignment-based one clearly improves hazard calibration and downstream accuracy.
Why not just apply standard missing-data correction, such as a logistic model of observation probability? That is close to what we do, and we include it as the text-only and uniform ablations. What is new is not inverse weighting itself. It is three things together: modeling deletion as a hazard over item age with eras of differing moderation, coupling those weights to temporal discounting, and building the benchmark that exposes the drift-versus-deletion confusion. If a plain logistic propensity model performs as well, that is a useful and publishable simplification.
Decisive Checks
Three results would establish the claim, and their failure would refute it.
First, in the constant-preference, moderation-change regime, NS-DPO on survivors should show a spurious drop in accuracy on the stable preference distribution. The survival-weighted estimator should recover most of the gap to the oracle.
Second, on WikiConv, reweighting snapshot data by the estimated survival probability should move the user-lifetime toxicity slope toward its full-history value. It should do so on the English split and also when the hazard is transferred to the Chinese split.
Third, in the dissatisfaction-deletion variant, survival weighting should restore a measurable share of DRIFT's advantage over plain DPO, the advantage that censoring erodes.
If the first check fails because a tuned discount already handles it, or the second fails because hazards do not transfer, the proposal reduces to a measurement paper about how moderated logs mislead drift estimators. That would still be a useful warning, but a narrower one.
References
- WikiConv: A Corpus of the Complete Conversational History of a Large Online Collaborative Community
- A Telegram Dataset of Propaganda and its Moderation
- Non-polar Opposites: Analyzing the Relationship between Echo Chambers and Hostile Intergroup Interactions on Reddit
- Tracking patterns in toxicity and antisocial behavior over user lifetimes on large social media platforms
- Right Now, Wrong Then: Non-Stationary Direct Preference Optimization under Preference Drift
- DRIFT: Learning from Abundant User Dissatisfaction in Real-World Preference Learning
- Aligning LLM Agents by Learning Latent Preference from User Edits
- Time-LLM: Time Series Forecasting by Reprogramming Large Language Models
- CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuning
- TSFM-Bench: A Comprehensive and Unified Benchmark of Foundation Models for Time Series Forecasting