Relation Subspaces as the Routing Key in Document-Level Relation Extraction

Document-level relation extraction reads a whole article and decides which relations hold between pairs of entities, for example that a person was born in a city mentioned three paragraphs later. Current systems build a single summary vector for each entity by pooling its mentions, and a single context vector for each pair. Only at the very last step does a relation-specific classifier decide whether "born in" or "employed by" holds. An entity like a politician carries many unrelated facets: party, birthplace, office, family. When all of its mentions, phrases, and sentences are pooled in one shared space, evidence for one relation is blurred by evidence for the others. This proposal moves the relation-specific step earlier. Each candidate relation gets a small learned subspace, a handful of directions in the encoder's vector space. That subspace decides which mentions, which short phrases, and which sentences count as evidence, at every level of a mention → phrase → sentence → document hierarchy.

The idea borrows from knowledge-graph embedding, where models like TransH and TransR project entities onto relation-specific hyperplanes or subspaces before comparing them. On FB15k, Hits@10 rose from 47.1% for TransE to 64.4% for TransH and 68.7% for TransR. Those projections were only ever applied to fixed entity vectors from graph triples, never to the many noisy mentions and phrases found in raw text. Our claim is that a projection only matters when it changes what gets selected during aggregation. We therefore make the subspace the thing that does the selecting, and we test whether its rank matters.

Core Claim

A linear projection commutes with averaging: projecting an averaged mention vector gives exactly the same result as averaging projected mentions. Adding a TransR-style projection on top of mean-pooled entities, as in HIN, therefore cannot fix the collapse caused by pooling. A projection adds something only when the weights used to pool are computed inside the relation's subspace.

RSMAN does a special case of this. It weights each mention by its dot product with a relation prototype vector, which amounts to a rank-1 projection followed by a linear score. A linear score on top of any projection collapses back to a single direction. So we score with subspace energy instead. Let $x \in \mathbb{R}^d$ be a mention, n-gram, or sentence vector. For relation $r$, let $U_r \in \mathbb{R}^{d \times k}$ have orthonormal columns built as $U_r = \mathrm{orth}(B A_r)$, where $B \in \mathbb{R}^{d \times m}$ is a basis shared across all relations and $A_r \in \mathbb{R}^{m \times k}$ is relation-specific. Then

$$ s_r(x) = \lVert U_r^\top x \rVert^2, \qquad \pi_r(x) = U_r^\top x . $$

Here $s_r$ measures how much of $x$ lies in relation $r$'s subspace, and $\pi_r$ is the $k$-dimensional projected vector passed upward. The energy score responds to any mix of the $k$ directions rather than one fixed direction. This lets a single relation such as "located in" be signalled by distinct facets: geographic nouns, prepositional phrases, administrative titles. Sharing the basis $B$ addresses the parameter blow-up that full per-relation matrices cause in TransR.

The hypothesis is falsifiable along one knob. At $k=1$ the model behaves like RSMAN's prototype attention. At $k=d$ with $U_r = I$ it becomes relation-agnostic pooling. If intermediate ranks beat both ends, with the gain concentrated on multi-mention entities and on evidence spread across sentences, then relation-specific routing is doing real work. If not, the claim is wrong.

Research Program

The encoder is a standard pre-trained transformer, kept identical to the ATLOP setup, with long documents split into overlapping chunks. Four aggregation tiers sit on top of it.

At the mention tier, each entity's mentions are pooled with softmax weights proportional to $s_r$. This produces a projected entity vector $\pi_r(e)$ for each candidate relation.

At the phrase tier, a bank of 1D convolutions slides over the transformer token states inside each sentence. These are fixed-width filters that act as learned n-gram detectors. Following SMS, we use widths 1 to 3. The n-gram features are pooled per sentence with weights from $s_r$, multiplied by the pair-specific token attention prior from ATLOP's localized context pooling. This keeps phrase selection specific to both the pair and the relation. Rather than gating the convolutions with fixed TransE vectors, as KCN does, we let the relation subspace select their outputs.

At the sentence tier, each sentence vector combines its pooled phrase evidence with its mean token state.

At the document tier, sentences are pooled with energy-based weights in the same subspace. The final logit for relation $r$ compares $\pi_r(e_h)$, $\pi_r(e_t)$, and the document evidence vector with a small bilinear head. Training uses ATLOP's adaptive-threshold loss.

Cost stays modest. Phrase and sentence projections are computed once per relation rather than once per pair. Only mention pooling is computed per entity and relation, and DocRED has 96 relations, so the overhead grows with $k$ times the relation count.

Beyond accuracy, the analysis asks three questions. Do the learned subspaces for related relations, such as "country" and "located in", overlap more than those for unrelated ones? Does mention-weight entropy drop as $k$ grows? Do the selected n-grams match DocRED's annotated evidence sentences?

Baselines

The naive reference point is a BERT-style encoder with mean-pooled entities and a bilinear classifier. HIN reports 53.20% F1 for this BERT-RE setup on the DocRED test set. HIN itself, at 55.60%, represents hierarchical aggregation without relation-specific pooling.

The strongest flat competitor is ATLOP, which reports 63.4 F1 on DocRED. It is the hardest baseline because its localized pooling already routes context to each pair, and its classifier is already relation-specific at the final step. For it to lose, early routing has to matter.

The strongest relation-specific mention model is RSMAN, which reports 70.95% F1 on DWIE when paired with SSAN and 1.0 to 1.5 F1 gains across backbones on DocRED. It is the direct rank-1 competitor.

On the biomedical CDR dataset, KCN reports 71.28% overall F1 but only 21.09% on inter-sentence pairs. ATLOP reports a 4.3 F1 gain over prior best methods on CDR, but no absolute ATLOP F1 for CDR was found on Lacuna, so we will rerun it. For LLMs, LMRC reports 60.69 F1 on DocRED with LLaMA-3.1-8B and 72.93 on Re-DocRED. We cite these numbers rather than rerun them.

For fairness, ATLOP, ATLOP+RSMAN, and our model are reimplemented on the same backbone, with BERT-base for DocRED and DWIE and SciBERT for CDR. Each gets the same tuning budget of 20 configurations and is run with five seeds. Two ablations isolate the key ingredient. The first sweeps $k$ over 1, 4, 16, and 64, plus the identity case. The second swaps energy scoring for linear scoring at the same rank, which should collapse toward RSMAN.

Beating ATLOP+RSMAN means a mean F1 gain larger than two seed standard deviations. In addition, at least half of that gain must come from inter-sentence pairs or from entities with three or more mentions.

Why not just…?

Why not just use RSMAN's prototype attention, perhaps with more heads? Multi-head prototype attention is the closest rival. It is a real threat, because $h$ linear heads span an $h$-dimensional space much like a rank-$h$ subspace. The difference is that energy scoring aggregates across directions nonlinearly, and the projected vector, not the original mention, is what travels up the hierarchy. We will run RSMAN with $h=k$ heads at matched parameter counts. If it ties our model, we will report that the routing benefit comes from relation-specific pooling, not from the geometry.

Why not just rely on ATLOP's transformer attention, with no convolutions or hierarchy? The transformer may already encode short n-gram triggers. The evidence against relying on it alone is mixed. SMS found that convolutional segments helped even on SpanBERT-large, but only for single sentences. HIN lost 2.81 F1 without hierarchical aggregation, but on a weaker baseline. The phrase-tier ablation, which removes the convolutions and keeps the subspace routing, settles this directly. We expect convolutions to matter mainly on CDR, where short trigger phrases like "induced" carry the relation.

Why not just fine-tune an LLM? LMRC shows that direct fine-tuning of LLaMA-2-13B reaches only about 39 F1 on DocRED. Even with candidate filtering, LLaMA-3.1-8B reaches 60.69, below ATLOP's 63.4, at far higher cost. Our model is a lightweight head that could also sit on top of LLM hidden states. We will test this only if budget allows.

Proposed Experiments

The main tables report F1 and Ign-F1 on DocRED, Re-DocRED, DWIE, and CDR.

The decisive check is a stratified breakdown along three axes: intra- versus inter-sentence pairs, sentence distance (with an explicit bin for 8 or more sentences, where HIN degrades), and the number of mentions per entity. We predict the gain grows with mention count, and is therefore larger on DWIE, which averages 1.98 mentions per entity versus 1.34 in DocRED. We also predict the gain grows with sentence distance. On CDR, the target is substantially closing KCN's 21.09% inter-sentence gap while matching ATLOP on intra-sentence pairs.

The rank sweep is the primary falsification test. A flat curve from $k=1$ to $k=64$ would mean the subspace idea adds nothing beyond prototype attention.

Finally, evidence-sentence recall on DocRED's annotated supporting sentences tests whether document-tier weights pick the right sentences. Subspace-overlap analysis tests whether the shared basis learns meaningful structure across relations.