Gate the Shared Case, Not the Pair: Dependency-Aware Data Selection for Cross-Jurisdiction Legal Retrieval

Legal search models learn from pairs, such as "this query case is relevant to that earlier case," and from short chains, such as "this case cites that statute, which defines the element at issue in a third case." In these datasets one document is reused across many training examples. A single case or statute can appear in hundreds of pairs and chains. When a model trained on one country's law is moved to another country, some of those reused documents carry legal concepts that do not exist, or mean something different, in the new legal system. Examples include a monetary threshold, a definition of intent, or a rule about who counts as the victim. Because such a document sits inside hundreds of training examples, the damage it does is multiplied by how often it is reused. Current methods that filter noisy training data decide example by example. They keep or drop each pair on its own.

We propose to make the keep-or-drop decision about the shared document itself: the case, the statute, or the bridge passage in a reasoning chain. The decision is made by a learned selection policy. Its reward comes from a small validation set in the target jurisdiction, and the reward is corrected for the fact that validation pairs are statistically coupled too. The claim is that the unit of selection should be the node of the dependency graph, not the edge. The claim also predicts where the gain should come from: document-level gating should beat pair-level gating by the largest margin on the most heavily reused documents. Those are the bridge nodes of multi-hop chains, and they are exactly where cross-jurisdiction concept conflicts concentrate.

Why the unit of selection matters

Learning theory for pairwise objectives already explains why reuse matters, but it has not been applied to data selection. Generalization Bounds for Regularized Pairwise Learning and A Uniform Convergence Bound for the Area Under the ROC Curve show that when each of $n$ samples appears in $O(n)$ pairs, perturbing one sample moves $O(n)$ loss terms. As a result, the standard independent-sample arguments fail. Chromatic PAC-Bayes Bounds for Non-IID Data makes this quantitative. It builds a graph whose vertices are training pairs, with an edge wherever two pairs share a sample. It then shows that the generalization bound scales with the graph's fractional chromatic number $\chi^*$, which is roughly the number of groups of mutually independent pairs one must split the data into. That paper motivates our proposal and supplies part of our method. It treats the dependency graph as fixed and given, whereas we treat it as something a selector changes.

When a selector drops a document, it deletes every pair incident to that document at once. This lowers the degree of the graph more than dropping the same number of scattered pairs would. For a selected set $S$ of pairs or chains with dependency graph $G(S)$, we use the chromatic view to define an effective training size:

$$n_{\text{eff}}(S) = \frac{|S|}{\chi^*(G(S))} \;\ge\; \frac{|S|}{\Delta(S)+1},$$

where $\Delta(S)$ is the largest number of other selected examples that share a document with any one selected example. The inequality is the standard fractional-coloring bound, and it gives us a tractable surrogate. Computing $\chi^*$ exactly is NP-hard, as the Chromatic PAC-Bayes paper notes, so we use $\Delta(S)+1$ or a greedy coloring in its place.

The same accounting applies to the reward. In target validation sets such as COLIEE case retrieval, many candidates hang off each query. A naive "validation gain after update" therefore averages dependent terms. Its variance is set by the number of queries, not the number of pairs. We believe this variance is one source of the reward noise and early collapse that the RL instance-selection literature reports.

Core Claim

Consider transferring a pairwise or chain-structured legal retriever from source jurisdictions to a target jurisdiction. We claim three things. First, hard selection of source documents beats hard selection of source pairs at equal budget. Second, the advantage grows with the reuse degree of the harmful documents. Third, a reward computed separately within each group of mutually independent validation pairs and then standardized across groups makes the selection policy stable where a naive reward does not.

Research Program

The target model is a two-stage legal retriever. It is a dense bi-encoder for first-stage matching, plus a two-hop chain scorer trained on (query case → cited statute or precedent → relevant case) chains. At test time it is used inside an iterative retrieve-then-reason loop in the style of IRCoT. IRCoT interleaves chain-of-thought steps with new retrieval queries on Wikipedia questions, where it gained 11–21 points of retrieval recall. We reuse its loop unchanged and only change what the retriever was trained on. The multi-hop setting is where the proposal applies, not what it contributes. It matters because bridge documents such as statutes and landmark precedents have the highest reuse degree. That high degree is what makes node-level gating matter.

The selector follows the actor-critic hard-gating design of RTNet. RTNet drops outlier-class source images in partial domain adaptation using a reward from a target-trained generator, and reaches 72.0% average accuracy on Office-Home against 64.5% for PADA and 70.5% for ETN. We change three things.

First, the action is taken on a source document, not on a training example. For each document the state includes its embedding, its degree in the pair-and-chain graph, the target model's average loss over its incident examples, and features of the statutes it cites.

Second, the reward is dependency-corrected. We greedily color the target validation dependency graph into independent classes. We then compute the metric change within each class and reward the mean change divided by its standard deviation across classes. This is a t-statistic, so the policy is paid only for improvements that survive the coupling.

Third, the objective adds a term favoring larger $n_{\text{eff}}(S)$. This discourages the trivial policy of keeping only a few very central documents.

Rewards are computed once per episode of a few hundred document decisions, not once per document. This keeps the inner-loop fine-tuning cost within a fixed budget. The analysis side is modest. We report the chromatic PAC-Bayes bound evaluated on the selected graph for each method. We then check whether bound tightness tracks measured target-jurisdiction error across selectors, or whether the bound is purely decorative.

Relation to closest work

Generative Context Pair Selection for Multi-hop Question Answering scores passage pairs by how well they regenerate the question. It shows that scoring pairs jointly resists distractors, with about a 1% drop against nearly 4% for discriminative selectors. It selects evidence at inference time and trains as if the pairs were independent. We select training data and account for the coupling.

Law-Match and NS-LCR improve Chinese case matching using cited articles as instruments or hand-written logic rules over 441 statutes, respectively. Both stay within one jurisdiction and treat pairs as independent. NS-LCR's rule curation is exactly the cost that a learned selector avoids.

ProMALex handles several jurisdictions by clustering them and branching adapters. It trains on all text and never removes incompatible material, so it is complementary. We use it as a backbone variant.

JPO uses reinforcement learning to shape an LLM's reasoning tokens within Chinese criminal law. Its policy acts on generated text, not on training data, and it is limited to single-jurisdiction, single-defendant cases.

Baselines

The naive reference point is training on the union of all source-jurisdiction pairs and chains plus the small target training split, with no selection. A second reference trains on target data only. Together these bound the value of transfer and show whether negative transfer exists at all in each source–target setting. If the union does not underperform some filtered subset, the whole question is moot, and we will report that.

The strongest existing selection competitor is pair-level hard gating. This is the RTNet actor-critic selector with the same state features and the same reward, acting on pairs instead of documents. It is the closest published mechanism, and it isolates our key ingredient directly. We also run pair-level continuous reweighting (a learned per-pair loss weight) to represent the soft-weighting family.

A cheap non-RL competitor is a similarity filter. It keeps source documents whose embeddings lie closest to the target jurisdiction's centroid, using the dataset-embedding-similarity idea from ProMALex applied at document granularity.

For the model backbone, we additionally report ProMALex-style jurisdiction adapters trained without selection.

Two ablations isolate the proposal's ingredients. One is document-level gating with the naive (uncorrected) reward. The other is document-level gating without the $n_{\text{eff}}$ term.

All selectors see the same candidate pool, the same target validation queries, the same number of target-model gradient steps, and the same number of reward evaluations. Hyperparameters are tuned with an equal budget of trials on a held-out validation split, disjoint from the query set used for reward. Beating pair-level gating means a gain on target-jurisdiction nDCG and F1 that exceeds the across-seed standard deviation over five seeds, and that is concentrated in queries whose gold chains pass through high-degree bridge documents.

For reference numbers, the file has no reported result on Lacuna for COLIEE case retrieval, so we will establish all COLIEE baselines ourselves. On related Chinese retrieval, the strongest cited figure is 55.1 MAP for Elem4LCR-I on LeCaRD-Elem, against 50.6 for BERT, as reported in An Element is Worth a Thousand Words. We use that model as the within-jurisdiction backbone check.

Why not just…?

Why not just filter source cases by embedding similarity to the target jurisdiction? This is the most serious threat. It is cheap, it already acts at document level, and it may capture most of the gain. Our reason to doubt it is that incompatible legal concepts often sit in texts that look topically identical. In credit-card fraud, whether the bank or the cardholder is the victim depends on property-rights rules, per Legal Analysis and Victim Identification of Credit Card Fraud Cases. Statutory damage thresholds also diverge while the surrounding wording stays the same, per Exploiting Contrastive Learning and Numerical Evidence for Confusing Legal Judgment Prediction. The controlled experiment below injects exactly such lookalike-but-wrong bridge documents. If the similarity filter matches the learned selector there and on COLIEE, we will conclude that the reward-driven policy is unnecessary. We will then report the document-versus-pair result using the similarity filter alone.

Why not just reweight pairs softly instead of dropping documents? A per-pair weight cannot represent "this document is wrong everywhere." It must learn the same low weight separately on each of the document's hundreds of pairs from noisy per-pair signal. The RTNet results also suggest that hard gating beats weighting-based adaptation when outliers are present. The continuous-reweighting baseline tests this directly on our data.

Why not just prompt a large general model and skip training-data selection? LawBench reports that even GPT-4 hallucinates or denies real statutes on civil-law tasks, so ungrounded prompting does not remove the jurisdiction problem. We nonetheless include a zero-shot LLM reranker over the same first-stage candidates as a reference point, so that readers can see the size of the gap on COLIEE rather than take it on faith.

Why not just assume pairs are independent and use the raw validation gain? That is precisely the naive-reward ablation. If it performs as well as the corrected reward, the dependency correction is unnecessary for selection. The theoretical point would then survive only as a diagnostic, and we will say so.

Proposed Experiments

The first experiment is a controlled test with known ground truth, run on MuSiQue and HotpotQA. MuSiQue is composed bottom-up from single-hop questions, so its chains are genuinely connected, and it shows a 30-point F1 drop for single-hop models relative to HotpotQA. We simulate a "foreign jurisdiction" by rewriting a chosen fraction of bridge passages so that a key relation flips while the surface text stays similar. We then vary the reuse degree of the corrupted bridges. Because we know which documents are harmful, we can measure how many of them each selector recovers, alongside downstream answer F1. Human performance on MuSiQue-Ans is 78.0 answer F1, which serves as a ceiling reference. The decisive prediction is that the gap between document-level and pair-level gating in harmful-document recall grows with bridge degree. A flat gap would falsify the mechanism.

The second experiment is real cross-jurisdiction transfer within English, so that jurisdiction is not confounded with language. We build citation-derived pairs and two-hop chains from the UK, India, EU, and ECHR portions of LexFiles as source data, with Canada held out. We evaluate on COLIEE Canadian case retrieval using nDCG, recall at k, and F1. As a second direction, we transfer from Chinese training data (LeCaRD, ELAM) and evaluate on the COLIEE Japanese statute task through a multilingual encoder. This direction is harder and reported separately.

The third experiment is a stability check. We compare reward variance, policy entropy over training, and the fraction of runs that collapse to keeping almost nothing, between the corrected and naive rewards. We also test whether the chromatic bound, evaluated on each method's selected graph, ranks the methods in the same order as their measured target error.

The proposal succeeds if all three of the following hold. Document-level gating beats pair-level gating on COLIEE at equal budget. The advantage in the controlled setting scales with bridge degree. The corrected reward reduces collapse. The proposal fails informatively if the similarity filter matches the learned selector, which would show that the node-versus-edge insight matters and the learned policy does not.