When to Retrain a Low-Resource Fake News Detector

Fake news detectors for languages like Malayalam, Tamil or Swahili are usually built once, on a small labeled set, by a team with little GPU time, and then left in place while the misinformation they face keeps changing: new hoaxes, new political framings, new Romanized spellings. Two questions decide how good the deployed detector is on today's posts: how often to retrain it on newly fact-checked posts, and how much of each retraining run to spend on hyperparameter search (trying learning rates and other settings) rather than on simply retraining with the old settings. This note proposes to answer both with a borrowed tool. Networking research measures "age of information," the time since the newest data reaching a monitor was generated, and has shown that sending updates as fast as possible is often worse than waiting until the age crosses a threshold. We treat the deployed detector as the monitor and each retraining job as an "update." Waiting until the detector's staleness crosses a learned threshold, and stopping retraining jobs that run too long, should give more accurate detection per GPU-hour than either fixed-schedule retraining or retraining whenever compute is free.

The opportunity matters because the low-resource setting has both a tight compute budget and fast topical drift, and the published systems ignore both. The Dravidian shared-task papers report one-shot results on fixed splits, and the hyperparameter literature treats tuning as a one-off cost rather than a recurring one that competes with freshness. If the threshold structure carries over, a small team gets a principled rule: when to retrain, and when re-tuning is worth the delay it causes.

Core Claim

Consider a detector that must be refreshed from a growing stream of fact-checked posts under a fixed budget of GPU-hours. A refresh launched at time $S_i$ uses labels available up to $S_i$, takes a random duration $Y_i$ that grows with the number $k_i$ of hyperparameter trials it runs, and is deployed at $S_i + Y_i$. Let $\Delta(t)$ be the age of the deployed detector's training data at time $t$, and let $g(\Delta)$ be the measured drop in macro-F1 on current posts for a detector whose data is $\Delta$ days old. The quantity to minimize is the long-run average accuracy loss

$$ \bar{L} = \lim_{T\to\infty} \frac{1}{T}\int_0^T \big[g(\Delta(t)) - h(k(t))\big]\,dt, $$

where $h(k)$ is the F1 gain from running $k$ tuning trials instead of reusing the previous configuration. The total GPU cost of all refreshes must stay within the budget. Our claim is that the optimal policy has the same shape found for compute offloading in Timely Cloud Computing: Preemption and Waiting. That paper chooses a waiting time before each heavy computation and a cutoff after which a slow job is cancelled. It shows that zero-wait is optimal only when the fixed processing overhead is large relative to the random part of the service time, with a sharp transition at $c=\sqrt{2}$ for shifted-exponential service. For detectors, we predict that a team should launch a refresh only when $g(\Delta)$ exceeds a threshold, and should cap tuning trials per refresh. We also predict that whether "retrain whenever idle" is optimal depends on the ratio of refresh time to drift speed. That ratio can be measured on real data.

What is new is not the use of retraining triggers, which practitioners already use. It is three things together. First, the staleness cost $g$ is measured empirically for a low-resource detector. Second, retraining delay and tuning effort are treated as one decision. Third, an online rule tunes the threshold without knowing the drift statistics, adapted from Age Optimal Sampling Under Unknown Delay Statistics. That paper adjusts a waiting threshold by stochastic approximation from observed delays and enforces a maximum sampling rate with a virtual "frequency debt" queue. We reuse that machinery, putting the GPU budget in place of the frequency limit and observed refresh durations in place of packet delays.

Relation to Prior Work

The age-of-information papers supply the method. Besides the two above, Age-optimal Sampling and Transmission Scheduling in Multi-Source Systems proves that with many sources sharing one channel, choosing whom to serve (the oldest first) separates from choosing when to sample, and that sampling follows a threshold. This motivates a multi-language extension: Malayalam, Tamil and Swahili detectors sharing one GPU, refreshed oldest-first. When to Preempt? constrains our design. It shows that an "always switch to the newest update" policy starves, completing no deliveries under high arrival rates. The analog is a team that keeps restarting a tuning sweep whenever new labels arrive and so never deploys anything; our cutoff must be paired with a rule against restarts. All of these papers model physical packets with known or stationary delay laws, and the file itself lists text and machine-learning pipelines as an unexplored domain. We differ by putting an empirically measured, possibly non-monotone accuracy-decay curve in place of linear age. We also take seriously the non-stationarity that the file flags as an open problem for Robbins–Monro tuning.

The compute-constraint literature motivates the tuning half. Jargon reports that it could not sweep hyperparameters for every French domain model, so differences between models may be tuning artifacts. Overtuning in Hyperparameter Optimization and Automatic Termination for Hyperparameter Optimization study when more search stops paying off. Our use of them differs in one respect: the stopping decision is scored against freshness lost while searching, not against validation loss alone.

The low-resource fake news work supplies the application and the detectors. Across Overview of the Shared Task on Fake News Detection in Dravidian Languages-DravidianLangTech@NAACL 2025, XLM-RoBERTa led both the binary and the five-class Malayalam tasks. CUET_Binary_Hackers reports that MuRIL, a model pretrained on Indian languages, won the multi-class task on code-mixed Malayalam. byteSizedLLM 2025 adds transliteration-aware masked-language-model pretraining so the model handles Romanized and native script. Beyond Tech gives SVM and Naive Bayes baselines that retrain in seconds. All of these evaluate a single static split. None asks how accuracy decays as posts drift, which is the gap this proposal targets. The caution in Harris et al., that translated data misses local framing, is why we refresh on organic, newly fact-checked posts and not on translated English data.

Research Program

The work has three stages. The first stage measures $g(\Delta)$. We need a time-ordered stream of labeled posts. If the DravidianLangTech Malayalam releases carry post dates, we order by date. If they do not (the file does not say), we build a small dated Malayalam stream from regional fact-checking outlets. As a fallback proxy, we cluster the shared-task data by topic and release clusters in sequence. For each detector family (TF-IDF with an SVM, MuRIL, XLM-RoBERTa) we train on data up to day $s$, test on each later window, and fit $g$ as a function of gap. We also measure the refresh duration distribution and the tuning gain $h(k)$ by running $k \in \{0,1,4,16\}$ trials of random search over learning rate, epochs and warmup.

The second stage solves the policy. Given the measured $g$, $h$ and duration distribution, we compute the optimal waiting threshold, trial cap and cutoff offline by fractional programming, as in Arafa et al. We then implement the online version, which learns the threshold from observed refreshes under a GPU-budget virtual queue.

The third stage replays the stream. Each policy runs over the same chronological sequence of labeled arrivals with the same GPU-hour budget. Every policy is scored on the posts arriving each day, using whichever detector is deployed at that moment.

Baselines

The naive reference is a static detector. XLM-RoBERTa is fine-tuned once on the earliest training window with standard settings and never refreshed. It shows how much drift costs at all.

The strongest existing detector setup is a fine-tuned XLM-RoBERTa, the shared-task leader per the 2025 overview, with MuRIL run alongside. The file gives no macro-F1 value: no reported result found on Lacuna for XLM-RoBERTa macro-F1 on the DravidianLangTech 2025 Malayalam fake news tasks. We therefore re-run it ourselves on the shared-task split before any streaming experiment, to confirm our setup is competitive.

The strongest scheduling competitors are three. First, fixed-period retraining (weekly, and the best period chosen in hindsight), which is what most teams do. Second, zero-wait retraining, which starts a new refresh as soon as the GPU is free, with default hyperparameters. Third, drift-triggered retraining, which refreshes when mean prediction confidence on incoming posts falls below a threshold chosen on the first window. All of them share our detector, data stream, GPU budget and tuning search space. The hindsight-best period gives the fixed schedule an advantage it would not have in practice.

Two ablations isolate the key ingredient. One runs our threshold policy with tuning switched off ($k=0$ always), which tests whether joint tuning allocation matters. The other removes the cutoff, which tests whether preempting long runs matters. Beating the competitors means lower time-averaged F1 loss than the best of them at equal GPU-hours, across at least three random stream orderings. A clearer win is matching their F1 at half the budget.

Why not just…?

Why not just retrain on a fixed weekly schedule? If drift is slow and refresh time is short relative to it, the theory itself predicts that a near-periodic policy is close to optimal. This is the regime where the $\sqrt{2}$-style transition sits on the "don't bother waiting" side. That is a genuine possible outcome, not a strawman. The stage-one measurements tell us in advance which regime the data are in. The replay against the hindsight-best period then measures how much is gained when drift is bursty, as it is around elections and health scares.

Why not just trigger retraining when the detector's confidence drops? This is the real threat. It already responds to drift, and confidence is cheap to monitor. Its weakness is that it ignores what the refresh costs. It cannot tell a 20-minute SVM refresh from a 6-hour XLM-RoBERTa sweep, and it gives no guidance on tuning effort. Also, confidence on Romanized or code-mixed posts may fall without any change in accuracy. We will also run a hybrid in which the confidence signal stands in for $\Delta$ inside our policy. If the hybrid matches our age-based policy, the contribution narrows to the cost-aware waiting and cutoff, and we will report it that way.

Why not just use a cheap classical model and retrain it constantly? Beyond Tech shows that SVM and Naive Bayes are viable on Malayalam, and the duration-free refresh removes the scheduling problem. The question is whether a fresh weak model beats a stale strong one. The same replay answers it, because the always-fresh SVM is included as a policy under the same budget.

Decisive Checks

The first check comes before anything else: does macro-F1 actually fall as training data ages? If $g$ is flat on the dated Malayalam stream, the premise fails and the project reduces to a negative result about drift in this domain. The second check is whether the offline-optimal threshold policy beats the hindsight-best fixed period and the confidence trigger at equal budget. The third is whether the online threshold learner converges close to the offline optimum within the stream length, given that drift is not stationary. The fourth, run on Malayalam, Tamil and Swahili sharing one GPU, is whether serving the stalest detector first beats round-robin, as the separation result predicts.