Suppose a language model must decide a court case written in Italian, and the most useful past cases to show it as examples are written in German or French. A standard approach embeds each whole judgment as a single vector and fetches the nearest past cases. We argue that this breaks down for a specific, measurable reason. Long legal documents are full of formulaic text: procedural headers, standard citations, and court phrasing that repeats across thousands of cases. That text is written in one language, so it pulls each document's vector toward other documents in the same language and away from cases that resemble it legally. The retriever then returns Italian neighbors because they are Italian, not because they are similar on the law. Filtering out this repeated text before building the search index should break that language pull. The filter itself needs no labels and does not depend on which language the text is in.
The proposal joins four lines of work and gives each a different job. Document noise supplies the mechanism. Semantic example retrieval, which means choosing few-shot examples for a prompt by nearest-neighbor search, is the application. The problem of evaluating models mostly in English motivates the setting: a three-language benchmark where English is absent. Domain-specific pre-training, meaning continued training of an encoder on legal text, is a variable we test rather than assume. If the claim holds, it tells practitioners when retrieval across languages actually works in specialized domains. It also gives a cheap fix that applies before any retriever training.
Core Claim
When full-text multilingual embeddings are used to retrieve examples from long technical documents, the main error is not weak alignment between languages. The main error is language-homophilous retrieval: neighbors come back in the query's language even when better legal matches exist in other languages. Formulaic text causes this, and the effect should grow with document length. Removing paragraphs that repeat throughout the corpus should do three things. It should raise the share of neighbors drawn from other languages. It should raise neighbor label purity, meaning the fraction of retrieved cases whose outcome matches the query's. And it should improve both a k-nearest-neighbor vote and few-shot prompting, with the largest gains for the low-resource language.
The nearest existing work sits on either side of this claim. XAMPLER trains a retriever that fetches English examples for queries in low-resource languages, using a language model's judgment of how helpful each example is. It works only on short topic-classification texts, where formulaic text does not exist. RAEvent strips narrative from Chinese legal cases by extracting legal events, which raised retrieval mean average precision from 48.68 to 60.37. That work is monolingual, depends on an event extractor trained for Chinese, and retrieves cases for search rather than prompt examples. PLJP rewrites case facts into structured triplets with a language model before retrieving precedents, again only in Chinese, and at the cost of extra model calls. Our filter needs no extractor, no rewriting model, and no training in the target language. Our question, whether noise causes retrieval to cluster by language, is one none of these papers ask.
Research Program
The main testbed is Swiss-Judgment-Prediction. It holds 85,000 Swiss Federal Supreme Court cases in German (50k), French (31k) and Italian (4k), each labeled approval or dismissal, with roughly 75–80% dismissals. One court applying one body of law in three languages gives a natural cross-lingual setting with no English pivot. The benchmark's authors already showed that Macro-F1 falls as documents get longer, which makes length a variable we can analyze.
The filter works as follows. Each judgment is split into paragraphs. Each paragraph gets a template score: the share of other cases in the corpus that contain a near-duplicate of it. Near-duplicates are found with character-shingle MinHash, a fast hashing method for spotting overlapping text, plus an embedding-similarity check. Duplicates are counted only within the same language. Paragraphs above a threshold are dropped, and the threshold is tuned on German and French development data only. The remaining paragraphs are embedded and pooled into one vector per case.
We compare this filter against two other ways of shortening a document. The first is truncation to the first $n$ tokens. The second is the sequential-window heuristic from the long legal document context-retrieval work, which showed that keeping nearby sentences beats selecting sentences by semantic similarity. That finding warns that aggressive pruning can destroy discourse structure, so we report results across a range of drop rates rather than at one chosen point.
We cross every document view with three encoders. The first is general multilingual XLM-R, the encoder that won on clinical text in the multilingual clinical named-entity study. The second is XLM-R given continued pre-training on the German, French and Italian training judgments. The third is that domain-adapted encoder further trained XAMPLER-style, with helpfulness labels scored by a language model on German and French training cases.
The question about domain pre-training is concrete. The clinical study found that domain models do not reliably beat large multilingual ones. The Swiss benchmark found native models win in German and French but not in Italian. We ask whether legal adaptation reduces language clustering by itself, or whether it learns the formulaic text better and makes clustering worse.
Each configuration is used two ways. One is a k-nearest-neighbor majority vote. The other is a few-shot prompt sent to one open-weight multilingual instruction model of about 7–13B parameters and one frontier API model. The query is an Italian (or French) case, and the example pool is restricted to the other two languages, or open to all three. Two diagnostics are reported in every setting, broken down by length bin: the cross-language neighbor share and neighbor label purity.
Baselines
The do-nothing reference is the share of dismissals in the test set, plus random selection of few-shot examples with a matched number of examples. Any method must clear these on Macro-F1 rather than accuracy, because dismissals dominate the label distribution.
The supervised ceiling is the Hierarchical BERT from Swiss-Judgment-Prediction. It splits a case into 512-token blocks and aggregates them with a BiLSTM. It reports Macro-F1 of about 68.5% in German and 70.2% in French. No result was found in this file for its Italian Macro-F1, so we will rerun it with the published setup and report the Italian number ourselves. It is the strongest competitor on outcome prediction because it uses every labeled case. We do not need to beat it with in-context prompting. We do need to report the gap honestly, and to show that the denoised k-NN vote narrows it in Italian, where training data is thin.
The strongest existing cross-lingual example retriever is XAMPLER, which reported 75.91% macro-accuracy on SIB200 against 74.61% for Multilingual E5. We re-implement it on our data with the same encoder backbone, the same budget of language-model scoring calls, and the same hyperparameter search, and we apply it to full-text cases. The proposal beats it if the template filter alone, without any retriever training, matches its Italian Macro-F1. It also beats it if the template filter combined with XAMPLER training exceeds it by more than the variation seen across random seeds.
The key ablation isolates the filter. We keep the encoder and the number of tokens fixed and swap the template filter for random paragraph dropping at the same drop rate. If random dropping recovers the gains, the effect comes from length, not from formulaic text.
On LexCLiPR, where queries in seven languages retrieve paragraphs from English judgments, the baseline is mDPR (multilingual dense passage retrieval). It reports 45.62% Recall@5% on seen concepts and 34.08% on unseen ones.
Why not just…?
Why not just translate everything into one language first? This is the real threat. Translating Italian queries into German makes every document the same language, which removes language clustering by construction. The clinical study found translate-test often trails direct cross-lingual transfer. LexCLiPR's zero-shot pipeline of translation followed by retrieval reached only 26.40% Recall@5%, but that is weak evidence for judgments thousands of tokens long. We will run translate-test with full-text retrieval as its own arm and report translation cost. If it matches the template filter, the clustering diagnosis still holds, but the fix becomes a matter of cost. Our filter's claim then shrinks to being cheaper. A third arm, translation plus filtering, tests whether formulaic text hurts even after everything is in one language.
Why not just take a majority vote over the retrieved cases and skip the language model? The legal judgment prediction evaluation found that a k-NN vote often beats prompting when retrieval is precise, and that weaker models degrade as more precedents are added. We therefore report the vote as a main result, not a strawman. The claim is about retrieval quality, which helps the vote as well. If prompting adds nothing over the vote, that is a finding we will report rather than hide.
Why not just feed full judgments to a long-context model? LitSearch found that indexing full body text hurts retrieval compared with indexing abstracts. The Swiss benchmark found that extending a flat model to 2,048 tokens did not beat hierarchical aggregation. Context length also does not fix which examples are retrieved, and that choice is where we locate the error. We still include a prompting arm that passes full precedents at matched cost.
Why not just fine-tune a legal encoder? That encoder is one of our arms. If domain pre-training alone removes language clustering, the filter is unnecessary, and the experiment will show it.
Proposed Experiments
The first decisive check is the diagnostic. Under full-text XLM-R, the cross-language neighbor share for Italian queries should fall below what equally accurate retrieval would produce. By "equally accurate" we mean the share expected if neighbors were chosen by label purity alone, estimated by permuting language tags. The gap should widen across length bins. If no such gap exists, the hypothesis is false and the project reverts to a plain comparison of methods.
The second check is the intervention. The template filter, compared with random dropping at the same drop rate, should raise both the cross-language neighbor share and label purity. It should also raise Macro-F1 for the vote and for few-shot prompting. We expect the effect to be larger for Italian than for German, because German queries have a large same-language pool, which makes same-language neighbors less harmful.
The third check is transfer to a setting with real relevance labels. We apply the filter to the paragraph pool in LexCLiPR and measure Recall@5% for seen and unseen concepts against mDPR. Gains on unseen concepts would suggest the filter removes surface cues that the retriever otherwise memorizes.
All results use five seeds, paired bootstrap confidence intervals, and identical budgets of retrieval and language-model calls across arms.
References
- XAMPLER: Learning to Retrieve Cross-Lingual In-Context Examples
- A Retrieval-Augmented Contrastive Framework for Legal Case Retrieval Based on Event Information
- Precedent-Enhanced Legal Judgment Prediction with LLM and Domain-Model Collaboration
- Swiss-Judgment-Prediction: A Multilingual Legal Judgment Prediction Benchmark
- A Simple but Effective Context Retrieval for Sequential Sentence Classification in Long Legal Documents
- Multilingual Clinical NER: Translation or Cross-lingual Transfer?
- LexCLiPR: Cross-Lingual Paragraph Retrieval from Legal Judgments
- A Comprehensive Evaluation of Large Language Models on Legal Judgment Prediction
- LitSearch: A Retrieval Benchmark for Scientific Literature Search