Do Legal Language Models Explain Themselves Better, or Only Look Like They Do?

Language models pre-trained on legal text predict court outcomes better than general models. On the Indian Supreme Court benchmark ILDC, a legally pre-trained BERT raises macro-F1 from 77.07% to 83.09%. Nobody has checked whether the explanations of these models improve too. Practitioners want those explanations in the form of the handful of sentences in a judgment that "made" the model decide. The standard way to get them is post-hoc occlusion: after training, hide one sentence at a time and see how much the prediction moves. This proposal asks whether domain pre-training makes such explanations more faithful (the hidden sentences really drove the decision) or only more plausible (they resemble what a lawyer would pick). It also asks how much of the answer is an artifact of the hiding operation itself.

That second question is the reason the comparison is not trivial. A model trained on legal text has a sharper sense of what a legal document looks like: its perplexity on Indian legal text is 5.25, against 25.76 for vanilla BERT. A judgment with a sentence replaced by mask tokens may look far stranger to the specialized model than to the general one. If so, an occlusion explainer will report larger "importance" for the specialized model partly because the specialized model is more surprised, not because it reasons more legally. Our core claim is testable. Comparisons of explanation quality across pre-training regimes are trustworthy only when the occlusion operation keeps the input inside each model's notion of normal text. We will measure how far each occlusion pushes each model off-distribution, and we will show whether the faithfulness ranking of general and legal models survives a switch to in-distribution sentence replacement.

What Exists and What Is Missing

Two papers motivate the work by exposing the gap. ILDC for CJPE built the ILDC corpus and the 56-case ILDCexpert set of expert-ranked rationale sentences. It trained a hierarchical XLNet+BiGRU model (chunk encoder plus recurrent aggregator) and extracted sentence rationales by occlusion. It found a "logic gap": expert agreement was low, with ROUGE-L around 0.44. Explanation Extraction from Hierarchical Classification Frameworks for Long Legal Documents (Ob-HEx, also Explanation Extraction from Hierarchical Classification Frameworks for Long Legal Documents) refined that occlusion with a score that weights sentence sensitivity by each sentence's contribution at higher levels of the hierarchy. It reports average gains of up to 3.93 metric points over the ILDC occlusion baseline for XLNet+BiGRU. Both papers use general-domain backbones. Both evaluate only agreement with experts, through ROUGE, BERTScore and Jaccard. Neither measures whether the extracted sentences actually carry the decision, and neither asks whether masked inputs remain sensible text.

The pre-training papers supply the models but never examine explanations. Pre-training Transformers on Indian Legal Text released InLegalBERT and InCaseLawBERT, both continued pre-training from BERT, and CustomInLawBERT, trained from scratch with a legal vocabulary. It also released the ISS rhetorical-role segmentation data, and it evaluates ILDC with a hierarchical BERT plus BiLSTM-attention head. It reports only classification. LEGAL-BERT adds a European/US legal model. It also constrains our fine-tuning, because it shows that standard recipes underfit legal models and that lower learning rates need a proper sweep.

Our evaluation method comes from ERASER, which separates plausibility from faithfulness. Plausibility is agreement with human rationales. Faithfulness has two parts: comprehensiveness, the drop in the predicted class probability when the rationale is removed, and sufficiency, the probability retained when only the rationale is kept. ERASER showed that plausible rationales can be unfaithful. The off-distribution concern comes from vision work. Manifold Integrated Gradients and MA-GIG show that attribution paths through unrealistic inputs produce noisy, unfaithful attributions. BioBART supplies the domain-text analogue, finding that shuffling sentences damages biomedical pre-training because discourse order carries meaning. We reuse a cheap off-distribution detector from Off-Manifold Collapse in Guided Protein Language Models, which filters generated sequences by their Mahalanobis distance to the training embeddings. That distance measures how many standard deviations an embedding sits from the training data, accounting for correlations between dimensions. We do not adopt the geodesic machinery of the vision papers, which assumes continuous inputs. Our unit of perturbation is a whole sentence, and our fix is substitution rather than path integration.

Research Program

We build a controlled set of models on ILDC that differ only in the chunk encoder: vanilla BERT, LEGAL-BERT, InLegalBERT, InCaseLawBERT and CustomInLawBERT. All share the hierarchical BERT-plus-BiLSTM-attention head of Paul et al., the same 512-token chunking, and the same fine-tuning sweep over learning rate and epochs, with three seeds each. Sentences are the unit of explanation. This choice matters, because sentence-level occlusion is identical across tokenizers, so the custom legal vocabulary does not distort the comparison.

For every model we extract rationales on ILDCexpert with both ILDC occlusion and Ob-HEx, under three ways of "removing" a sentence. Masking replaces the sentence with mask tokens, as prior work does. Deletion drops the sentence entirely. In-distribution replacement substitutes a sentence drawn from a different case. That sentence must have the same rhetorical role (facts, arguments, statute, ratio) and similar length, with roles assigned by a labeler trained on ISS. Replacement keeps the document's structure and register intact while removing the sentence's specific content, which is the textual counterpart of staying on the data manifold.

For each operator and model we compute two quantities. The first is an off-distribution score: the Mahalanobis distance of the perturbed document's chunk embeddings from that model's training-set chunk statistics, averaged over occlusions. The second is the full ERASER pair of plausibility and faithfulness. The program then asks three things. Does domain pre-training raise plausibility and faithfulness? Does masking push legal models further off-distribution than vanilla BERT? Does the faithfulness ranking of models change when off-distribution operators are replaced with in-distribution ones?

Baselines

The naive reference point is vanilla BERT in the same hierarchical head, explained by ILDC-style mask occlusion. This is the pipeline that produced the logic gap. Two pieces of context frame it: the original XLNet+BiGRU explanations reached ROUGE-L of about 0.44 according to ILDC for CJPE, and that paper's ILDC-multi classification baseline was 77.07%.

The strongest existing explainer is Ob-HEx from Prasad et al.. It reports an average gain of up to 3.93 metric points over ILDC occlusion on long explanations (ranks 1–3) with XLNet+BiGRU. It is the strongest competitor because it already exploits the hierarchy. We will rerun it on every backbone with the authors' scoring, rank cutoffs and metrics, so our plausibility numbers are directly comparable to theirs.

The strongest predictive model is InLegalBERT. It reports 83.09% macro-F1 on ILDC-multi in Paul et al.. We must first reproduce this figure within seed variance, so that faithfulness differences are not caused by an undertrained model. For faithfulness on ILDC there is no reported result found on Lacuna for comprehensiveness or sufficiency with any model. ERASER's own faithfulness numbers are not in the file either. Our faithfulness baseline is therefore one we establish ourselves, using the vanilla-BERT mask-occlusion pipeline.

Two further baselines guard the faithfulness numbers. Random sentence selection at matched rationale length gives a floor for comprehensiveness and sufficiency. Raw attention weights from the aggregator supply the attention baseline that ERASER found unfaithful.

The ablation that isolates our key ingredient holds the model and the explainer fixed and changes only the removal operator: mask, delete, or role-matched replacement. A further ablation replaces role-matched sentences with random-role sentences, to test whether matching the rhetorical role matters or whether any fluent sentence suffices.

Beating the baselines means two things. On the scientific question, we would show that the model ranking under masking differs from the ranking under replacement by more than seed variance, or that it does not. On the practical side, a legal model plus Ob-HEx under replacement would exceed vanilla BERT's comprehensiveness by more than the seed standard deviation without losing ILDCexpert ROUGE-L.

Why not just…?

Why not just run Ob-HEx on InLegalBERT and report ROUGE against the experts? This is the obvious study, and we include it as one cell of the design. On its own it measures only plausibility. ERASER's central finding is that plausibility can rise while the model ignores the selected text. A higher ROUGE for InLegalBERT would also be confounded by its higher accuracy. We therefore add faithfulness, and we compare models at matched validation accuracy, using early checkpoints of the stronger model.

Why not just delete sentences instead of building role-matched replacements? This is a real threat. Deletion is simpler, and a hierarchical model tolerates variable document length. However, deletion also shortens the document and can drop a whole chunk boundary, which may itself be unusual for the model. We run deletion as a full condition. If the off-distribution scores for deletion and replacement are similar, and the two operators agree on the model ranking, we will recommend deletion and report the replacement machinery as unnecessary.

Why not use attention weights or gradients, which come for free? ERASER reports that attention is often less faithful than post-hoc methods such as LIME. Attention Meets Post-hoc Interpretability argues that attention describes internal routing rather than input importance. We include aggregator attention as a baseline so this claim is tested on legal models rather than assumed. Gradient methods over subword tokens would also make the comparison depend on each model's tokenizer.

Why not ask a large general model to explain the verdict? That would explain a different model's reasoning, not the classifier being deployed. The post-hoc NLP survey warns that free-text explanations can be detached from the computation they claim to describe. This alternative answers a different question, so we leave it out of scope.

Proposed Experiments and Decisive Checks

The main experiment crosses five encoders, two explainers and three removal operators on ILDCexpert, with three seeds per encoder. For each cell we report expert ROUGE-1, ROUGE-2, ROUGE-L, BERTScore and Jaccard, following Ob-HEx. We also report comprehensiveness and sufficiency at rationale sizes of 1–3 and 1–5 sentences, and the mean off-distribution score.

The first decisive check is whether, under masking, the legal models' occluded documents sit measurably further from their training statistics than vanilla BERT's do. If they do not, our confound hypothesis is wrong, and existing occlusion comparisons can be taken at face value. The second check is whether the faithfulness ordering of models measured with masking survives under replacement. A flip, or a collapse of the gap between general and legal models, would show that current occlusion-based comparisons are partly measuring surprise rather than reasoning. The third check repeats plausibility and faithfulness at matched classification accuracy, to separate "better model" from "better-explained model."

A small expert review closes the gap noted by Ob-HEx. A lawyer will review the sentences that legal and general models disagree on, for 20 cases, and judge whether each sentence is legally relevant. This tests whether any rise in faithfulness tracks substantive legal content or procedural boilerplate.

As a cross-domain replication, we will run the operator ablation on ERASER's Evidence Inference task, comparing BERT with a BERT continued on PubMed abstracts, the corpus BioBART used. Faithfulness is measured with ERASER's own scorer. There is no reported result found on Lacuna for Evidence Inference faithfulness to compare against. This replication tests whether the operator effect is specific to legal text or a general property of domain-pre-trained models.