Reading or Remembering? Counterfactual Probes for Long-Tail Relation Extraction at Scale

When a large language model labels the relation in "Marie Curie was born in Warsaw" as place of birth, it may be reading the sentence, or it may simply know where Marie Curie was born. Standard few-shot relation extraction benchmarks cannot tell these apart. They are built from Wikipedia, news, and PubMed text that modern models have almost certainly seen during pretraining. This proposal builds matched test variants of those benchmarks. In one variant, the real entities are swapped for plausible invented ones, so memory cannot help. In another, the sentence asserts a relation that contradicts what the model "knows," so memory actively misleads. We then measure how much of a model's accuracy survives, separately for common ("head") and rare ("tail") relations and for model sizes from about 6B to 70B parameters and beyond.

The central prediction is specific and falsifiable. On tail relations, the accuracy that larger models gain over smaller ones will come mostly from recalling the specific entity pair rather than from reading the evidence. The memorization share will therefore grow with scale on the tail and stay flat or shrink on the head. If this holds, published long-tail gains are partly an artifact of benchmark contamination, and the scaling story for relation extraction needs to be retold. If it fails, we will have the first direct evidence that large models genuinely read rare relations from text. The second half of the proposal tests a cheap fix. It fine-tunes on counterfactual sentences so that the text, not memory, decides the label, and checks whether this fix still works at 70B.

Core Claim

Existing contamination work shows the ingredients but has never combined them for relation extraction. Snoopy correlates pretraining term counts in The Pile with few-shot accuracy for GPT-Neo and GPT-J-6B, and argues that scale helps partly by memorizing rare terms. Its analysis covers arithmetic, sentiment, and QA, not relational triples, and stops at 6B. Back to Square One shows that RoBERTa-large exceeds 60% on the Winograd Schema with the candidate entities hidden, which exposes artifacts. Hiding entities removes information, however, so it cannot separate "reads the context" from "recalls the fact." DyVal sidesteps contamination by generating fresh reasoning problems from graphs. It works only for formal math, logic, and algorithm tasks. Unveiling Causal Reasoning uses post-cutoff news, where Claude 3 Opus falls below 70% on the hard subset. Fresh text changes topic and style as well as novelty, so it confounds contamination with distribution shift.

Our claim is that a minimally edited counterfactual test, which keeps the sentence and changes only the entity pair, isolates reliance on memory. We need three quantities per model, relation bucket, and scale. The first is $A_{\text{orig}}$, accuracy on the original sentence. The second is $A_{\text{sub}}$, accuracy after swapping in type-consistent invented entities. The third is the conflict-follow rate $C$, the fraction of counterfactual sentences (text asserts relation $r'$, world knowledge says $r$) on which the model outputs the memorized $r$ instead of the textual $r'$. We define memorization reliance as

$$R = A_{\text{orig}} - A_{\text{sub}},$$

which is how much accuracy disappears when the entity pair can no longer be recalled. We also report a context-free accuracy $A_0$, where the model sees the entity pair but no sentence, as an upper bound on what memory alone can deliver. The hypothesis is that $R$ and $C$ on tail relations increase with parameter count, and that $R$ tracks the pretraining co-occurrence count of the entity pair.

This framing takes up a tension the file surfaces. Language Models as Knowledge Embeddings attributes C-LMKE's strength on degree 0–1 entities in FB15k-237 to rich textual descriptions. Our design lets us ask whether that strength survives when the descriptions refer to entities the model has never seen.

Research Program

Building the probes. We start from four long-tail sources in the file. FewRel provides clean human-annotated sentences. The FewRel 2.0 PubMed/UMLS split and its none-of-the-above ("NOTA") setting add a biomedical domain shift and queries that match no candidate relation. The NYT distant-supervision relations with fewer than 100 training instances come from Hierarchical Relation Extraction. Wiki-One and NELL-One from One-Shot Relational Learning provide KG-side sparsity.

For each test sentence we produce three variants: the original, an entity-substituted version, and a counterfactual version. Invented entities are sampled to match the entity type expected by the relation, so a birthplace becomes a pronounceable town-like name. They are checked to have near-zero counts in The Pile. Counterfactual variants swap in a real entity of the right type that contradicts the known triple, and adjust the relation phrase when the target relation changes. Annotators confirm on a sample that each edited sentence still determines its label from text alone.

Tail-ness is measured on two axes. The first is the relation's training-instance count, which the NYT and one-shot KG sets supply. FewRel is class-balanced, so on FewRel we rely on the second axis: the pretraining co-occurrence count of the entity pair. We compute this exactly in The Pile, following the approach of Snoopy.

The scale sweep. Models run in a fixed prompted few-shot format. GPT-Neo and GPT-J-6B are included because their pretraining corpus is known, so $R$ can be regressed on exact co-occurrence counts. LLaMA 2 at 7B, 13B, and 70B supplies the within-family scale axis. One frontier API model serves as a ceiling. We also fine-tune a subset with LoRA at 7B, 13B, and 70B, because the long-tail methods in the literature are trained rather than purely prompted.

The fix. We fine-tune with a mixture of original, substituted, and counterfactual training sentences, all labelled by what the text says. The test is whether this lowers $C$ and $R$ without hurting $A_{\text{orig}}$, and whether its benefit shrinks or grows from 7B to 70B. The file flags that interventions designed on 2B–8B models often weaken at 70B, as in Breaking Bad Tokens. We add an analysis motivated by Geometric Signatures of Compositionality, which found that nonlinear intrinsic dimension tracks meaning while linear dimension tracks surface form. We measure the intrinsic dimension of entity-pair representations on original versus substituted inputs. If memorized pairs occupy a lower-dimensional, lookup-like region, that would give a geometric signature of recall. This analysis is exploratory. The behavioral metrics above are what decide the claim.

Baselines

The naive reference is prompted few-shot accuracy on the unedited benchmark, $A_{\text{orig}}$. This is the number the field would normally report, and every finding is stated relative to it.

The strongest trained few-shot relation classifier in the file is BERT-PAIR from FewRel 2.0, which reports 80.31% accuracy in 5-way 5-shot with a 50% NOTA rate. Prototypical Networks in FewRel report 69.20% in 5-way 1-shot and 75.55% in 10-way 5-shot. On the KG side, GMatching reports an MRR of 0.200 on NELL-One and 0.20 on Wiki-One. We retrain BERT-PAIR and ProtoNet on the same splits and evaluate them on all three variants. They are the right comparison because a BERT-base encoder has seen far less pretraining text. If its $R$ is near zero while large models show large $R$, contamination explains the gap; if BERT-PAIR shows similar $R$, the effect is not scale-driven. No reported result was found on Lacuna for LLaMA-scale or frontier models on FewRel, NELL-One, or Wiki-One, so we generate those numbers ourselves.

For an autoregressive extraction system trained on relation data, we run GenIE under the same probes.

For the fix, the baseline is standard LoRA on original data only. It uses the same rank, the same number of optimizer steps, and a matched hyperparameter search budget of the same number of trials per method. The key ablation trains on substituted sentences only, with no counterfactuals. This isolates whether the conflict between memory and text, rather than mere entity novelty, is what teaches the model to read. The fix beats its baseline if it reduces $C$ by a margin exceeding bootstrap confidence intervals at every scale, while keeping $A_{\text{orig}}$ within one point of plain LoRA.

Why not just…?

Why not just run a membership-inference test such as Min-K% Prob on the test sentences? Membership inference asks whether a sentence was seen. We ask whether the prediction depended on the memorized fact. A model can have seen the exact FewRel sentence yet classify it by reading, or never have seen it yet know the entity pair from a thousand other pages. The research file also notes that such tests struggle to detect subtle leakage at terabyte scale. We will run Min-K% Prob as a secondary signal and report its correlation with $R$. A weak correlation would itself show that sentence-level detection misses fact-level reliance.

Why not just build a fresh post-cutoff relation dataset, as CausalProbe-2024 did? This is a real alternative, and we include a small post-cutoff news set as a check. Its weakness is confounding. New text brings new topics and new entities together, so a drop cannot be attributed to missing memory rather than distribution shift. Unveiling Causal Reasoning cannot separate these either. Minimal edits keep syntax, domain, and relation phrasing fixed. If fresh-data drops and $R$ agree in size, the simpler method is vindicated for auditing, though it still cannot measure the conflict-follow rate.

Why not just mask the entities, as in Back to Square One? Masking tells us what can be done without entities. It does not tell us whether the model relies on knowing them, and masked sentences are unnatural inputs that may depress accuracy for unrelated reasons. We run masking as an ablation alongside substitution. If masking and substitution give the same $R$, the cheaper masking protocol suffices.

Why not just prompt the biggest model with chain-of-thought and declare the long tail solved? A high $A_{\text{orig}}$ from a frontier model is exactly the number our probes question. DyVal found that 33.8% of GPT-4's failures were unsubstantiated answers. Unveiling Causal Reasoning found that chain-of-thought did not stop models defaulting to high-probability clichés. We include chain-of-thought prompting at every scale; if it drives $C$ to near zero, then prompting, not fine-tuning, is the remedy, and we will say so.

Proposed Experiments

The first decisive check is the scale-by-tail interaction. We fit $R$ against log parameter count, tail bucket, and their interaction on NYT tail relations, Wiki-One, and pair-frequency buckets of FewRel. The claim holds if the interaction is positive and significant, meaning tail $R$ grows with scale while head $R$ does not. It fails if $R$ is flat across scale or larger on the head.

The second check uses GPT-Neo and GPT-J, where the corpus is known. The claim predicts that $R$ rises monotonically with the entity pair's Pile co-occurrence count, and that $A_{\text{sub}}$ does not depend on that count.

The third check revisits the FewRel 2.0 puzzle the file highlights: is the collapse under domain shift and NOTA a failure of adaptation, or does it expose inflated in-domain accuracy? We compare $R$ on Wikipedia FewRel with $R$ on the PubMed split. The inflation account predicts that $R$ is larger in-domain, where entity pairs are heavily represented in pretraining. It also predicts that in-domain $A_{\text{sub}}$ approaches the PubMed $A_{\text{orig}}$.

The fourth check tests the fix at 7B, 13B, and 70B, against plain LoRA and the substitution-only ablation, reporting $C$, $R$, and $A_{\text{orig}}$ with bootstrap intervals. The result that would most change practice is a fix that works at 7B but loses most of its effect at 70B. That would mean larger models' reliance on memorized relational facts is harder, not easier, to override.