Training Citation-Graph Classifiers to Survive What the Database Left Out

Most automated tools for mapping science, such as topic classifiers, trend detectors and literature-synthesis pipelines, are trained on one bibliographic index: arXiv, PubMed, the ACL Anthology or Web of Science. Each index is a curated slice. It omits regional and non-English venues, preprints and conference series outside its remit (A bibliometric analysis of global research output on network meta-analysis; Profiling COVID-19 Genetic Research; Panaroma of microglia in traumatic brain injury). A paper from outside the index looks different to a graph model. It is different partly because of its words, but mostly because the database records only a few of its citations. Its true neighbors were never indexed. A model trained only on well-connected, well-covered papers has never seen a paper with its context cut away. It will misclassify exactly the work the bibliometric literature says is already undercounted.

We propose to train for this failure directly. For each training paper, an adversary searches for the worst plausible "coverage cut": it removes the citations to whole venues or neighbor groups, within a budget. The model is then trained on hidden representations that interpolate, and slightly extrapolate, between the paper's full-context view and its censored view, while the paper keeps its own label. Two sentences suffice to explain the idea. Instead of mixing two different papers, as manifold mixup does, we mix a paper with an adversarially gutted copy of itself. Doing so teaches the classifier that losing database coverage should not change what a paper is about.

Core Claim

Single-source database bias is usually treated as a disclaimer added at the end of a paper (A bibliometric analysis of metonymy in SSCI-indexed research). For a learned model, though, it is a specific and structured distribution shift. Papers outside the index, or at its edges, arrive with their neighborhood truncated in blocks, because indexes drop whole venues and repositories, not random edges. Their text is also somewhat shifted. Our claim is that robustness to this shift can be trained from a single source by combining three things. The first is a worst-case search over structured censoring operations. The second is a small latent perturbation for vocabulary shift. The third is manifold mixup along the direction from the full view to the censored view. We further claim that this beats both random censoring and generic adversarial or mixup regularizers on the accuracy measured on papers held outside a simulated index, without hurting accuracy inside it.

Concretely, let $f$ be a graph encoder producing a hidden vector $h_v = f(v; G)$ for paper $v$ in citation graph $G$. Let $c$ be a censoring operation drawn from a set $\mathcal{C}$ of block removals, such as deleting all edges to one venue cluster. The block size is capped by a budget. Let $\delta$ be a latent perturbation with $\|\delta\| \le \epsilon$. The adversary chooses $$ (c^*, \delta^*) = \arg\max_{c \in \mathcal{C},\, \|\delta\|\le\epsilon} \; L\big(g(f(v; c(G)) + \delta),\, y_v\big), $$ where $g$ is the classifier head and $y_v$ is the paper's label. Training then uses the mixed state $$ h^{mix}_v = \lambda\, h_v + (1-\lambda)\,\big(f(v; c^*(G)) + \delta^*\big), \qquad \lambda \sim \text{Beta}(\alpha,\alpha) \text{ shifted to } [-\beta, 1], $$ and scores it against the unchanged label $y_v$. Values of $\lambda$ below zero push past the censored view, which lets the model reach configurations sparser than any cut the adversary actually found.

Relation to Prior Work

Several papers supply the method. Manifold mixup moves interpolation from raw inputs to hidden states (Manifold Mixup: Encouraging Meaningful On-Manifold Interpolation; Manifold Mixup: Better Representations by Interpolating Hidden States). That move matters here, because averaging two citation graphs is meaningless. MixUp as Directional Adversarial Training shows that mixup is perturbation toward another sample while the original label is kept. We take that reading literally and replace "toward another sample" with "toward this sample's worst-case censored self." The label-preserving view is what makes this legitimate: a paper's topic is invariant to which of its citations a database happened to index. Learning Gradient-based Mixup towards Flatter Minima (FGMix) motivates the extrapolation. It argues that interpolation is trapped inside the convex hull of the training data, so it extrapolates with mixing weights outside $[0,1]$ and uses a discriminator for stability. We borrow the extrapolation but keep it short and anchored to a real censored view instead of a learned weight network.

Other papers combine adversarial search with latent mixup, and our proposal must be distinguished from them. Manifold Adversarial Augmentation for Neural Machine Translation adds gradient perturbations and pairwise mixup in Transformer hidden layers for translation. Its perturbations are unstructured, and it says nothing about graphs or coverage. Single Domain Generalization with Model-aware Parametric Batch-wise Mixup (MPBM) is the closest in spirit. It uses adversarial queries plus attention-weighted batch mixup to generalize from one image domain, reaching 54.5% average accuracy on PACS. Its adversary works in feature space and mixes across samples. It has no notion of a structural operation such as losing neighbors, so porting it to graphs would perturb node features while every citation stayed intact. FIXED constrains the design in two ways. It shows that mixing across samples shrinks representations toward the mean and blurs class boundaries, and its remedy needs multiple labeled domains, which a single index does not provide. Self-mixing avoids the cross-class blurring because both endpoints carry the same label. Target-Oriented Single Domain Generalization needs a text description of the target domain. We cannot write one for papers a database never saw.

On graphs, Batch Virtual Adversarial Training for Graph Convolutional Networks (BVAT) is the strongest adversarial-training precedent on citation networks. It perturbs node features and handles overlapping receptive fields, but it evaluates only random in-distribution splits and never perturbs the edges. Harnessing Explanations (TAPE) supplies strong frozen text features for text-attributed citation graphs. STRIDE reports temporal out-of-distribution gains on ogbn-arxiv and OGBN-Mag. Neither one tests transfer across database coverage. Mixed Graph Contrastive Network and Local Mixup show that node-embedding mixup helps on graphs and that mixing distant points intrudes on other classes. Self-mixing is the extreme case of locality.

The motivating evidence comes from bibliometrics. AtlasNLP finds that 79.2% of country–task pairs in the ACL Anthology have no documented dataset and that the represented country is missing for 74% of records. Coverage gaps are large, and they cannot be filtered out after the fact.

Research Program

The first deliverable is a coverage-shift benchmark with ground truth. This piece is missing in the literature, and the method cannot be judged without it. We start from a richer graph and apply an explicit indexing policy. The result is an "indexed" subgraph for training and an "unindexed" set of papers for testing. Each unindexed paper is seen only through its edges into the indexed part, which is exactly how a paper from outside an index looks to a deployed model. OGBN-Mag records papers, authors, institutions and fields, so it allows two kinds of policy. Venue-based policies keep a chosen set of venues, mimicking an index's publisher agreements. Institution-based policies drop papers whose authors are all affiliated with a held-out set of institutions, mimicking regional exclusion. On ogbn-arxiv, we use the temporal axis, with tape-arxiv23 as an extra held-out set, and a subject-cluster policy. These splits are deliberately label-correlated: some fields lose more coverage than others. This correlation is what makes worst-case training differ from random training.

The second deliverable is the training method above. The search over censoring operations is relaxed to a continuous mask over venue or neighbor blocks. We take one gradient step on that mask, round it to the top blocks within the budget, and take one projected step on $\delta$. The encoder sees the censored graph in one extra forward pass. We expect roughly twice the cost of standard training. That is comparable to SSMix's reported slowdown and below O-BVAT's roughly threefold overhead. The encoders are a GCN on bag-of-words features, to match BVAT's setting, and a graph neural network on TAPE-style frozen language-model features, to test whether the gain survives strong text.

The third deliverable is an analysis of where the gain comes from. We bucket test papers by the fraction of their true neighbors removed by the indexing policy, and we plot accuracy against that fraction. The method should flatten this curve. A method that only raises the curve's intercept is not addressing coverage.

Baselines

The naive reference point is a GCN, and the same graph neural network on language-model features, trained by plain empirical risk minimization on the indexed subgraph. It is evaluated inside the index and on unindexed papers, and the gap between those two numbers is the quantity we try to close. The strongest existing adversarial method on citation graphs is GCN with O-BVAT (Batch Virtual Adversarial Training for Graph Convolutional Networks). It reports 83.6% accuracy on Cora, 74.0% on Citeseer and 79.9% on Pubmed under standard in-distribution splits, against 81.5%, 70.3% and 79.0% for the plain GCN. Those numbers do not transfer to our setting: no reported result found on Lacuna for any method on cross-coverage or index-exclusion splits. We will therefore first reproduce BVAT's Cora number to validate our implementation and then rerun it on our splits.

Two generic baselines need adapting to graphs. MPBM (Single Domain Generalization with Model-aware Parametric Batch-wise Mixup) represents adversarial batch mixup for single-source generalization; its reported image result is 54.5% average accuracy on PACS. The Manifold Adversarial Augmentation recipe (Manifold Adversarial Augmentation for Neural Machine Translation), meaning a random-layer gradient perturbation plus pairwise hidden mixup, stands for the "both ingredients, unstructured" combination. We also include plain manifold mixup between nodes. The TAPE-feature model (Harnessing Explanations) serves as the strong-features reference, run both with and without graph edges.

Three ablations isolate the key ingredient. The first replaces adversarial block censoring with random block censoring of matched size, and the second uses random per-edge dropping at a matched edge budget. Together they test whether "adversarial" and "block-structured" matter. The third trains directly on the censored view without mixing ($\lambda = 0$), which tests whether interpolation adds anything over plain augmentation. A further ablation removes the extrapolation by restricting $\lambda$ to $[0,1]$.

Every method gets the same encoder, the same indexed training graph, the same number of tuning trials, chosen on a validation set drawn only from inside the index, and the same wall-clock budget. Wall-clock matching gives cheaper baselines extra epochs. Beating a baseline means higher accuracy on unindexed papers, averaged over five seeds and all indexing policies, by more than the seed standard deviation. In-index accuracy must stay within one point of the empirical-risk baseline.

Why not just…?

Why not just randomly drop edges during training? This is the most serious threat. If the excluded papers' missing citations were scattered at random, random dropping would cover the shift and the adversary would add only cost. Our splits are built so that coverage loss correlates with field and venue, as the bibliometric evidence suggests real coverage loss does (A bibliometric analysis of the trends, topics, and findings of research publications on asynchronous and synchronous online language learning). The matched-budget random-censoring ablations settle the question directly. If they match the full method on unindexed and worst-field accuracy, we will report that the useful contribution is the benchmark plus a finding: block censoring suffices, and adversarial search is unnecessary.

Why not just drop the graph and classify papers from their text? If citations are the corrupted signal, a text-only model on strong language-model features avoids the problem. It also gives up the citation signal, which BVAT and TAPE show to be valuable inside an index. The edges-removed TAPE baseline measures this trade-off. The method is only worth having if it beats both the full-graph model and the text-only model on unindexed papers. If text-only wins, that is a meaningful negative result for graph-based bibliometric tools.

Why not just reweight the under-covered papers inside the index? Upweighting sparsely connected training papers is cheap, and it targets the same population. It cannot create the specific configuration of a paper whose entire home venue is missing, because such papers do not exist in the training graph. We include degree-based importance reweighting as an extra baseline in the bucketed analysis. Reweighting should help most in moderately censored buckets and fade in the most heavily censored ones, where we expect extrapolation to matter.

Why not just pool several databases? Pooling databases is the real fix when it is possible. Schema mismatches, deduplication and licensing make it rare in practice. Even pooled indexes still share blind spots, which AtlasNLP's 74% missing-attribution figure illustrates. Our method targets the common case where only one source is available.

Proposed Experiments

The decisive experiment trains on each simulated index, on OGBN-Mag with venue and institution policies and on ogbn-arxiv with subject and temporal policies plus tape-arxiv23. It then reports three numbers: accuracy on unindexed papers, accuracy on the worst field, and the slope of accuracy against the fraction of neighbors removed. The claim holds if the full method beats O-BVAT, the MPBM-style port and random block censoring on all three, at matched compute, while keeping in-index accuracy. The claim fails if random block censoring comes within seed noise.

A second check tests whether worst-case training overfits the indexing policy it was trained against. We train with censoring along venue blocks and test on institution-based exclusion, and the reverse. Gains that survive this crossing indicate robustness to coverage loss in general rather than to one kind of cut.

A third, smaller check uses real data. We train a task classifier on well-attributed ACL Anthology records from AtlasNLP's high-coverage countries. We then evaluate on its human-curated Gold records from the countries that appear least often in AtlasNLP's own tallies, and ask whether the training gains on the simulated splits carry over to a naturally occurring coverage gap. This check is small and noisy, and we will present it as directional evidence only.