Knowledge graphs that change over time, such as political event databases, biomedical literature graphs and product-review networks, are increasingly extended by models that propose new time-stamped facts. These models are good at ranking candidates but not good enough at picking the single correct one, so a human expert still has to approve what gets written in. Expert time is the scarce resource. This proposal asks a narrow question: when the facts carry timestamps, which few candidates should the expert check? Our answer is that the most valuable check is often not the one the model is least sure about. It is the one whose confirmation brings the model's picture of a stale part of the graph back up to date. A confirmed fact with a timestamp changes the "time since last evidence" that the model's temporal encoding feeds on for every pending prediction that touches the same entities. One answer can therefore repair many predictions.
Concretely, we build a query-selection rule on top of models that already encode elapsed time between events, which is what a temporal positional encoding does: it maps timestamps or time gaps to vectors so the network can tell "five minutes ago" from "five months ago". The rule scores each pending candidate by its own risk of error plus the expected improvement it would cause in neighboring pending decisions once its verified timestamp is inserted. We test whether this beats uncertainty sampling and simple novelty routing under a fixed budget of expert answers, measured by the precision of what actually gets committed.
Core Claim
Under a fixed query budget, selecting expert checks by staleness-weighted propagation value yields higher precision of auto-committed facts than three alternatives: entropy-based uncertainty sampling, routing all never-before-seen edges to the expert, and the same procedure run on discretized snapshots. The gain should come specifically from the interaction between verified timestamps and the model's relative-time encoding. If the gain survives when the temporal encoding is removed, the claim is wrong.
Relation to Prior Work
Two lines of work motivate the gap. Interactive curation tools such as CleanGraph let a human accept or reject link predictions and errors through subgraph views and scoped edits, but they operate on static graphs with no notion of event time. Human-in-the-loop handling of knowledge drift (TRCKD) shows how much leverage targeted queries give: about 1.5 structural queries per run recover 5–10% micro-F1 after a taxonomy changes. It does, however, detect change only by comparing a reference window against a sliding window, and it treats time as coarse batches. Both papers support the premise that a few well-chosen questions matter. Neither says which questions to ask when evidence arrives continuously.
The closest work is THiGER-A, Link prediction for hypothesis generation: an active curriculum learning infused temporal graph-based approach. It selects uncertain training pairs by entropy with k-means++ diversity on a snapshot-based temporal graph, and has biologists review 402 predictions offline, finding 70% of high-probability ones plausible. We differ on three points. THiGER-A uses active selection to reduce negative-sample bias during training, whereas we select queries to decide what gets committed online. It discretizes time into snapshots, whereas we exploit continuous relative-time encodings. Its expert review is post hoc, whereas our expert's answers are fed back as time-stamped evidence.
The method is supplied by continuous-time encoders. TiPNN encodes paths in a unified history graph using the gap between query time and event time, and reports running 67–80% faster than snapshot methods. L-STEP treats each node's positional trajectory as a signal filtered in the frequency domain, with linear cost. Both are fully automated. L-STEP also admits that it degrades under abrupt structural change and for nodes with no history, and those are exactly the regions where we expect expert checks to pay most.
One benchmark result constrains the design. Temporal Graph Benchmark (TGB) shows non-learning heuristics often win: a moving average reaches NDCG@10 0.559 versus TGN's 0.315 on tgbn-reddit. Any triage policy must therefore treat "this edge has happened before" as a first-class signal, not something to beat.
Research Program
The system has three parts. The first is a base temporal link predictor that produces scores for pending candidate facts $(s, r, o, t)$, where $s$ is the subject entity, $r$ the relation, $o$ the object entity and $t$ the timestamp. We use TiPNN on temporal knowledge graphs and L-STEP on TGB link prediction, with scores calibrated by temperature scaling on a held-out stretch of time. The second is a commit rule: candidates whose calibrated probability $p_c$ exceeds a threshold are written to the graph automatically, and the rest are held. The third is a query selector that spends the expert budget.
The selector scores a pending candidate $c$ by
$$V(c) = \min(p_c, 1-p_c) + \lambda \sum_{c' \in N(c)} \mathbb{E}_{y_c}\big[\Delta \mathrm{acc}(c' \mid y_c, t_c)\big].$$
Here $N(c)$ is the set of pending candidates that share an endpoint with $c$ inside the encoder's receptive field. The term $\Delta \mathrm{acc}(c' \mid y_c, t_c)$ is the change in the expected correctness of the commit decision on $c'$ after inserting $c$ with answer $y_c$ at time $t_c$. The expectation over $y_c$ uses $p_c$, and $\lambda$ trades off direct error risk against propagation.
The propagation term is computed by actually re-scoring the neighbors under both hypothetical answers. That is affordable because L-STEP's per-node update is linear and TiPNN's path computation is local. Staleness enters implicitly. A neighbor whose endpoints have not been touched for a long time sits far out on the encoder's elapsed-time axis, so a fresh verified timestamp moves its score most. Batching follows TRCKD-style targeting, with a few questions per round and each answer inserted before the next selection.
After each round, verified facts, and also verified rejections stored as negative evidence, are inserted with their timestamps. The encoder state is updated without retraining.
Baselines
The naive reference is no queries at all: commit by threshold only. This tells us how much any expert budget buys. A second naive reference is random selection under the same budget.
The strongest existing selector is entropy uncertainty sampling with k-means++ diversity, as in THiGER-A. We run it on the same base model, candidates, calibration and budget. Only the selection rule differs, which isolates the selection question from THiGER-A's snapshot encoder. For context, THiGER-A reports binary F1 0.97 on its PubMed food-biomedical task and 70% plausibility on 402 reviewed pairs. Neither is a budgeted-query result, and we found no reported result on Lacuna for budgeted active verification on ICEWS18, GDELT or TGB.
The strongest heuristic competitor is novelty routing. It auto-commits candidates that EdgeBank-style memory has seen before when the model agrees, and spends the budget on unseen edges in order of model score. TGB's moving-average result (NDCG@10 0.559 versus 0.315 on tgbn-reddit) is why this baseline is serious. It gets the same budget and the same commit threshold.
Two ablations isolate the key ingredient. The first sets $\lambda = 0$, leaving pure risk-based selection on the temporal model. The second keeps the full rule but replaces the continuous elapsed-time encoding with monthly snapshot features, so verified timestamps cannot change staleness at a finer grain. We claim a win only if the full rule exceeds every baseline and both ablations on committed precision at matched recall, across at least three budget levels, with non-overlapping 95% bootstrap intervals over five temporal splits.
Why not just…?
Why not just use uncertainty sampling, as THiGER-A does? Entropy picks the candidate the model is least sure of. It ignores the fact that one answer about a long-dormant entity changes the elapsed-time input for every pending fact involving that entity. The $\lambda = 0$ ablation is exactly this comparison on equal footing. If it matches the full rule, the propagation idea adds nothing and we will report that.
Why not just send never-seen edges to the expert and trust repeats? This is the real threat, since TGB shows recurrence heuristics are hard to beat. We expect it to do well on low-novelty streams and to waste budget on high-novelty ones, where most unseen edges are individually low-value. We therefore stratify results by TGB's surprise index. If novelty routing wins on high-surprise datasets too, our method's niche shrinks to low-surprise data and we will say so.
Why not just bin time into snapshots and run static active learning? TiPNN reports that snapshot methods fragment history and run slower. More to the point here, snapshots cap how much a single verified timestamp can update a neighbor's state. The snapshot ablation tests this directly.
Why not let a language model verify instead of a human? CleanGraph reports hallucination and factual errors from LLMs in specialized domains. We treat an LLM as an extra noisy oracle condition rather than a replacement, and measure committed precision when its answers are used in place of ground truth.
Proposed Experiments
The main runs use ICEWS18 and GDELT with TiPNN, and tgbl-wiki and tgbl-review with L-STEP. Streams are replayed in time order. Ground truth from later observations serves as a simulated oracle, at budgets of 0.1%, 0.5% and 2% of pending candidates per day. To address the open problem of noise-free oracle assumptions, we repeat every run with oracle flip rates of 5% and 15%.
The primary metric is precision of auto-committed facts at matched recall, plotted against queries spent. Secondary metrics are MRR gain on future test events per query and the average number of neighbors whose commit decision flipped to correct per answer.
The decisive check is a dose-response test. If the mechanism is real, the advantage of the full rule over the $\lambda = 0$ ablation should grow with the staleness of the queried entities' neighborhoods and should vanish under the snapshot ablation. A gain that does not track staleness would mean the improvement comes from something other than the temporal encoding.
References
- CleanGraph: Human-in-the-loop Knowledge Graph Refinement and Completion
- Human-in-the-loop handling of knowledge drift
- Link prediction for hypothesis generation: an active curriculum learning infused temporal graph-based approach
- Temporal inductive path neural network for temporal knowledge graph reasoning
- Learnable Spatial-Temporal Positional Encoding for Link Prediction
- Temporal Graph Benchmark for Machine Learning on Temporal Graphs