Stratified Adversarial Alignment for Cold-Start Transfer Between Temporal Knowledge Graphs

A temporal knowledge graph stores timestamped facts such as (Country A, threatens, Country B, 2014-03-02). The usual task is forecasting: given everything up to today, predict tomorrow's missing subject or object. Recent "vocabulary-agnostic" models have shown that a reasoner trained on one such graph can be applied to a completely different one, with no shared entities, relations, or time granularity. These models describe relations by how they co-occur and replace calendar dates with relative positions. A model trained on daily political news can then score queries on 15-minute event streams or on yearly biographies. The open question is what still separates such a transferred model from one trained on the target graph. This note proposes that a measurable part of that gap is nuisance information: graph density, how often facts simply repeat, and event rate. The structural representation absorbs these properties from the source graph, and they are not part of the reasoning that transfers.

The proposal tests this in two stages. First, a diagnostic. Can a simple classifier tell source queries from target queries using the transferred model's internal representations, and does that separability predict how much accuracy is lost on each dataset pair? Only if it does, the second stage applies adversarial domain adaptation, meaning an encoder trained to fool a classifier that tries to identify which graph a representation came from. The adaptation is stratified: the classifier only compares source and target queries that share the same observable predictive statistics, such as recent repetition and entity activity. This targets the known failure of adversarial alignment, which erases features the task needs. The intended regime is cold start, where the target graph has only a handful of snapshots of history.

Core Claim

Unconditional adversarial alignment is the wrong tool for temporal knowledge graphs. The statistics that differ most between graphs are also the strongest predictors of the answer. Whether a fact recurred in the last few snapshots is the clearest example, and it is why local "copy" windows matter so much in these models. Forcing source and target representations to be indistinguishable would therefore delete the signal that does the predicting.

The claim is that conditioning the domain classifier on these observable, label-free query statistics removes only the residual source-specific skew. In the cold-start regime, this should close more of the transfer gap than fine-tuning on the few available target snapshots, explicit statistical normalization, or pooling several source graphs. If the stage-one diagnostic shows that domain separability does not track the transfer gap, the claim is false and the adversarial stage should not be built.

Relation to Prior Work

The closest work is FITTER. It builds a fully inductive temporal reasoner from relational interaction graphs, relative sinusoidal time encodings, and fused local and global context. Across 15 cross-dataset scenarios it recovers 89–96% of in-domain specialist performance. FITTER supplies the backbone and motivates the proposal: its invariance comes entirely from architectural choices, and it has no mechanism to correct density or edge-formation skew in the source graph. We keep its architecture and add a training objective that it lacks.

Two lines of work use the word "adversarial" differently. Wasserstein Adversarial Learning based Temporal Knowledge Graph Embedding trains a Gumbel-Softmax generator to produce hard negative facts. T3DM also uses a GAN-style hard-negative sampler, and adds test-time training on an LSTM that predicts which entities will appear. Both operate on fixed entity lookup tables within a single graph, and neither aligns representations across domains. They therefore constrain the design (a transductive method cannot be a cross-vocabulary baseline) rather than supply it. T3DM is still useful because it documents that relation frequencies and tone drift between training and test periods.

On the adaptation side, DASTNet supplies the method. It uses a gradient reversal layer (an identity in the forward pass that flips the sign of the domain classifier's gradient on the way back) to learn city-invariant traffic dynamics, and reports roughly 42% lower error than standard models with one day of target data. That cold-start result is the main precedent for our regime. However, DASTNet's graphs are static road networks with numeric signals and a single whole-city domain label. Its unconditional alignment is exactly what our stratified variant is meant to improve on.

The concern that alignment can distort what the label depends on, which motivates stratification, appears in this literature as negative transfer. The sensitivity reports in FAN-Net, where perturbing frequency components beyond a small threshold strips semantic boundaries, and in Episodic Training for Domain Generalization, where the weighting of episodic losses is fragile, both warn that alignment strength must be controlled. The episodic paper also notes that simple data pooling often beats more complex domain generalization methods. We take that as a constraint and include pooling as a baseline.

Research Program

Stage one: diagnostic. We train FITTER on each source graph and extract query representations for both source and target queries. Using these, we fit a linear probe to predict domain identity. We report probe accuracy before and after regressing out a vector $z$ of observable query statistics:

  • the number of times the query's subject–relation pair had any object in the last $w$ snapshots;
  • the log-degree bucket of the subject;
  • the relative gap since the subject's last event;
  • the number of distinct relations in the target window.

The key quantity is the rank correlation, across dataset pairs, between residual probe accuracy and the measured transfer gap. The transfer gap is in-domain specialist MRR (mean reciprocal rank) minus transferred MRR, both measured by us.

Stage two: stratified adversarial training. We train only if the stage-one correlation is clearly positive. Let $\phi(q)$ be the encoder's representation of query $q$, $d\in\{\text{source},\text{target}\}$ its graph of origin, and $D$ a domain classifier that sees both $\phi(q)$ and $z(q)$. Training optimizes

$$\min_{\phi}\max_{D}\; \mathcal{L}_{\text{rank}}(\phi;\text{source}) \;-\; \lambda\, \mathcal{L}_{\text{dom}}\big(D(\phi(q), z(q)), d\big),$$

where $\mathcal{L}_{\text{rank}}$ is the usual ranking loss on source facts, $\mathcal{L}_{\text{dom}}$ is the cross-entropy of the domain classifier, and $\lambda$ weights alignment.

This is implemented with a gradient reversal layer. Because $D$ is given $z$, the encoder is only pushed to make domains indistinguishable within strata of $z$. Retaining $z$ itself costs the encoder nothing, so the predictive repetition and activity signals survive alignment. Target queries enter $\mathcal{L}_{\text{dom}}$ only. They come from the first $k$ snapshots of target history, and their answers are never used in this objective. The parameter $\lambda$ is ramped up on the schedule standard in gradient-reversal work and is selected on a source-only validation split, so no target labels leak into tuning.

Dataset pairs. We use three kinds of pairs:

  • ICEWS14 to ICEWS18: same schema, pure temporal drift.
  • ICEWS05-15 to GDELT: daily to 15-minute granularity, with roughly 2.7 million facts on the target side.
  • ICEWS14 to YAGO11k and Wikidata12k: dense political events to sparse yearly biographies with 10 and 24 relations.

Cold start is simulated by giving each method only the first $k \in \{0, 1, 5, 30\}$ target snapshots before forecasting the remainder. FITTER's full GDELT test split takes about 21 hours on an A100. We therefore evaluate GDELT on a fixed, stratified random subsample of test queries, shared across all methods, and report its size and confidence intervals.

Baselines

The naive reference point is FITTER applied zero-shot: trained on the source and run on the target without any use of the $k$ target snapshots beyond reading them as history. This is the number every method must beat. The per-pair cross-domain MRRs needed here were not in the file; FITTER's summary gives only the 89–96% recovery range. We rerun it on identical query sets.

The strongest existing competitor is FITTER fine-tuned with its own ranking loss on the same $k$ target snapshots, with early stopping on a held-out slice of those snapshots. It is the strongest because in a temporal graph the target history is itself labeled data, which makes it the obvious thing a practitioner would try. It gets the same snapshots, the same GPU budget, and the same number of hyperparameter trials as our method.

The multi-source pooled baseline trains FITTER on the union of all non-target graphs (for example ICEWS14, ICEWS18, and YAGO11k when the target is GDELT). This tests the episodic paper's warning that pooling is hard to beat.

The normalization baseline rescales the nuisance statistics directly. It resamples GDELT to daily snapshots and reweights source training queries so that their distribution of $z$ matches the target's. This is importance weighting with no adversary.

Two ablations isolate the key ingredient. The first is unconditional adversarial alignment, the same gradient-reversal setup but with $D$ blind to $z$, as in DASTNet. The second is a shuffled-domain-label control, where $D$ is trained on random labels, to rule out gains that come only from extra regularization noise. An in-domain specialist provides the upper reference. On the in-domain side, the file reports MRR 0.471 on ICEWS14 and 0.521 on ICEWS05-15 for LLM-DA with a TiRGN backbone, and MRR 56.32 on WIKI for CoPET. These are reference ceilings that use target labels or text, not competitors.

Beating the strongest baseline means a higher MRR than fine-tuned FITTER at $k \le 5$ on at least two of the three cross-schema pairs, with non-overlapping 95% bootstrap intervals. The stratified variant must also beat the unconditional variant on the same pairs. Otherwise the conditioning idea, which is the contribution, has not been shown.

Why not just…?

Why not just fine-tune the transferred model on the target's history? This is the most serious threat, and for large $k$ we expect it to win. With 30 snapshots of GDELT, there is plenty of supervised signal. Our argument applies only to the smallest histories. With one to five snapshots, ranking-loss fine-tuning sees few distinct facts and can overfit to whatever happened to repeat. The adversarial term uses the same snapshots only to constrain the geometry of the representations. The $k$-sweep settles this directly. If fine-tuning matches or beats stratified alignment at $k=1$, the proposal reduces to "fine-tune," and we will report it that way. A combined variant (alignment followed by fine-tuning) is also run, since the two are not exclusive.

Why not just normalize the nuisance statistics explicitly? If the source skew is fully described by $z$, importance reweighting and granularity resampling should remove it without a min-max game, which is known to be unstable. The stage-one diagnostic answers this before any adversarial training. If domain identity stops being decodable once $z$ is regressed out, explicit normalization suffices and the adversary has nothing left to remove. The adversarial method is justified only if residual decodability remains and tracks the transfer gap.

Why not just use test-time training as in T3DM? T3DM raised TTransE's MRR on ICEWS18 from 8.36 to 11.98, but its auxiliary task predicts marginals over a fixed entity table. That task cannot even be defined when target entities are disjoint from source entities. The natural vocabulary-agnostic version is self-supervised link prediction on recent target snapshots, which is the fine-tuning baseline above. So T3DM's idea is covered rather than dismissed. LLM rule rewriting as in LLM-DA is excluded from the head-to-head comparison: it reasons over verbalized relation sequences and abstracts away entities. We note this as a scope limit rather than a defeat of that approach.

Decisive Checks

The first check comes before anything is trained adversarially. Across the dataset pairs (the three described here, extended to FITTER's other pairings where available), residual domain decodability must show a positive rank correlation with the transfer gap. If it does not, we stop and publish the negative diagnostic, which is itself informative about where transfer loss in inductive temporal reasoners actually lives.

The second check concerns the claimed mechanism. Stratified alignment should leave the probe's ability to recover $z$ from the representation intact, while unconditional alignment should degrade it. The accuracy difference between the two should be concentrated on queries with high recent repetition, which are the queries unconditional alignment is predicted to damage.

The third check is the cold-start curve. The advantage over fine-tuning should be largest at small $k$ and shrink as $k$ grows. A flat advantage at every $k$ would suggest the gain comes from regularization rather than alignment, and the shuffled-label control would then indicate which explanation holds.