Shrinking Covariances Toward a Reference That Moves With the Data

Many methods that work on learned features rely on an inverse covariance matrix. Mahalanobis out-of-distribution (OOD) detectors use one, and so do transferability scores and generalized-eigenvector feature extractors. In principle these methods have a useful property: they ignore how the feature axes are scaled, rotated or sheared, so an invertible linear map applied to the features leaves their output unchanged. That property fails once the covariance has to be regularized. When there are about as many feature dimensions as samples, the sample covariance is singular or badly distorted. The standard fix is to blend it with a multiple of the identity matrix, which is Ledoit-Wolf shrinkage, and the identity is tied to one particular choice of coordinates. The regularized method then depends on arbitrary preprocessing choices after all.

This note proposes a simple fix and gives a reason why no other kind of fix exists. When there are fewer samples than dimensions, no estimator built only from the small sample can be both invertible and fully linear-equivariant. Equivariant here means that if the data are transformed by $A$, the estimate is transformed to $A\hat\Sigma A^\top$. So invariance has to come from outside the sample. The remedy is to shrink toward a reference covariance estimated from a larger pool that lives in the same feature coordinates. In many deep-feature pipelines such a pool comes for free. For per-class covariances in OOD detection, it is the pooled covariance of all in-distribution training features. For small-target transferability scoring, it is the covariance of the source or unlabeled features. We call the estimator reference-whitened shrinkage. It whitens the data with the reference, shrinks toward the identity in that frame, and maps back. The result is invertible, equivariant under every invertible linear map, and computable in closed form.

Core Claim

Let $S_k$ be a small-sample covariance, for example that of class $k$ with $n_k$ samples in $p$ dimensions. Let $R$ be a reference covariance estimated from a much larger sample of the same features. The estimator is

$$\hat\Sigma_k = (1-\alpha_k)\,S_k + \alpha_k\,c_k\,R, \qquad c_k = \mathrm{tr}(R^{-1}S_k)/p .$$

Both $\alpha_k$ and $c_k$ are computed in the whitened frame $z = R^{-1/2}x$. In that frame the formula is exactly Ledoit-Wolf shrinkage toward a scaled identity, and $\alpha_k$ is given by the usual analytic Ledoit-Wolf formula applied to the whitened samples. Every quantity involved is unchanged when all data undergo the same invertible linear map, so $\hat\Sigma_k$ transforms congruently and every downstream Mahalanobis or Rayleigh-quotient score is exactly invariant.

A nonlinear version replaces the single blending weight with the nonlinear shrinkage that Robinson et al. use in An Improvement on the Hotelling T² Test Using the Ledoit-Wolf Nonlinear Shrinkage Estimator. Here it is applied to the generalized eigenvalues of the pair $(S_k, R)$, which are themselves invariant.

The claim has three parts. The first is a small theorem: for $n \le p$, any linear-equivariant function of the sample alone is singular on the complement of the sample span. The argument is that one can choose a transformation $A$ that fixes the sample points but acts arbitrarily on that complement. The second part is that reference-whitened shrinkage is the natural invariant way around this obstruction. The third, which the experiments must settle, is that it helps in practice. It should outperform both identity-target shrinkage and diagonal simplifications on OOD detection and small-data transferability, and its gains should persist when the features are deliberately re-coordinatized.

Relation to Prior Work

The motivating evidence comes from work showing that invariance and regularization are currently in tension. Quantifying the Variability Collapse of Neural Networks builds a linear-invariant collapse metric using a Moore-Penrose pseudo-inverse of the raw total covariance. It is invariant but unregularized, and it is left exposed exactly when $n \ll d$. Newer is Not Always Better shows the opposite trade. The H-score, a transferability score, becomes competitive once its covariance is Ledoit-Wolf shrunk toward the identity, but that choice gives up invariance to rescaling and shear.

A Bayesian Nonparametric Perspective on Mahalanobis Distance for OOD Detection is the closest competitor. It shows that class covariances of ViT-B-16 features are heterogeneous, so the usual shared ("tied") covariance is wrong. It also shows that full per-class covariance models collapse from overfitting. Its answer is a coupled-diagonal model, which discards cross-feature correlations and with them invariance. Our estimator sits between these options. Each class keeps its full covariance, pulled toward the tied covariance by a data-chosen amount, and none of the three existing choices are needed.

The method-supplying work is the shrinkage literature. Ledoit-Wolf and its nonlinear variant supply the formulas we apply in the whitened frame. Robust Shrinkage Estimation of High-Dimensional Covariance Matrices supplies a Tyler-type robust iteration. Normalizing samples in the $R$-whitened frame, rather than in raw coordinates, makes that iteration invariant as well. Generalizing Analytic Shrinkage for Arbitrary Covariance Structures shows that identity-target shrinkage underregularizes spiked spectra. Whitening by a good reference should flatten those spikes before shrinking, which gives us a testable prediction.

Affine Invariant Covariance Estimation for Heavy-Tailed Distributions is conceptually closest on the estimation side. It whitens with its own running estimate, then clips, over sample-split iterations. Its guarantees need $n$ on the order of $d$. Self-whitening cannot escape the singularity obstruction when $n<p$, and an external reference can. Semantic Similarity Covariance Matrix Shrinkage also shrinks toward an externally built target, but its target comes from text embeddings with no invariance motive. Finally, Discriminative Features via Generalized Eigenvectors (GEM) is invariant but breaks when a class second-moment matrix is singular. Shrinking each class matrix toward the pooled second moment keeps its generalized eigenproblem both solvable and invariant.

Research Program

The work has three parts.

The first part is analytical. We will prove the singularity obstruction and prove equivariance of the estimator, including the nonlinear and robust variants. We will then bound the extra error that comes from using an estimated $R$ with $N$ samples instead of the true one. The expected result is that this error enters roughly as $\sqrt{p/N}$, following standard sample-covariance concentration, so a reference sample much larger than $p$ makes it negligible relative to the $\sqrt{p/n_k}$ error being corrected.

The second part is algorithmic. We will compute $\alpha_k$ from whitened samples and handle $p>n$ through low-rank identities in the style of Ibrahim et al., so the score costs $O(n^2 p)$ plus one factorization of $R$.

The third part is empirical, across three uses of covariances. Class-conditional Mahalanobis OOD detection is the primary test. Small-target transferability scoring, through a shrunk H-score and a shrunk collapse index, is secondary. GEM on MNIST and TIMIT subsampled so that $n_k<p$ is a sanity check.

Baselines

The naive reference points are the unshrunk estimators: the tied-covariance Mahalanobis score, the relative Mahalanobis score, and the raw pseudo-inverse used by Xu and Liu's collapse index. All are exactly invariant, so they show what invariance buys without regularization.

The strongest existing method for OOD is the coupled-diagonal Gaussian mixture model of Linderman et al. On SSB Hard near-OOD with ViT-B-16 features, they report 80.98% AUROC, against 80.03% for the relative Mahalanobis score and 78.97% for standard Mahalanobis. We will rerun all three from their described setup, using the same ViT-B-16 penultimate features, the same in-distribution training set, and the same OOD splits. We will first confirm we reproduce their numbers. Beating the baseline means exceeding 80.98% AUROC on SSB Hard, averaged over seeds with a confidence interval that excludes it, without degrading far-OOD AUROC. Matching it while remaining invariant would be a weaker but still reportable result.

For transferability, the strongest competitor is Ibrahim et al.'s Ledoit-Wolf-shrunk H-score. The file gives only "up to 80% absolute gain in correlation" and no per-dataset number, so there is no reported result on Lacuna for the H-score with shrinkage on a specific benchmark. We will rerun it ourselves on the same models, target subsets and fine-tuning ground truth. Beating it means higher rank correlation with fine-tuned accuracy at small target sizes. For the collapse index, Xu and Liu report a Pearson correlation of 0.975 to 0.995 with transfer accuracy at full data. Our question is whether that correlation holds up better under subsampling with the shrunk version.

For GEM, the reference results are 96 MNIST test errors (deep GEM) and 40.86% TIMIT frame error, both from Karampatziakis and Mineiro at full data. Our comparison is at deliberately reduced per-class sample sizes, against GEM with identity-ridge regularization.

The key ablation holds the shrinkage formula fixed and swaps only the target. The candidates are a scaled identity (standard Ledoit-Wolf), $\mathrm{diag}(R)$ (scale-invariant only), $R$ itself, and a mismatched $R$ estimated from a different backbone's features after a random linear map. The mismatched reference keeps invariance but removes the information in the target. This ablation separates "the target is informative" from "the target is equivariant."

Why not just…?

Why not just z-score the features and use ordinary Ledoit-Wolf? Standardization removes per-axis scale but not correlations or shear. It is a real threat, though, because practical features may mostly differ in scale. The target-swap ablation settles this directly. If the $\mathrm{diag}(R)$ target matches the $R$ target on unperturbed features, full invariance is only a robustness property, and we will report it that way. The perturbation test below shows whether that robustness matters.

Why not just use regularized discriminant analysis, blending each class covariance with the pooled one and then with the identity? The class-to-pooled blend is essentially our estimator with a hand-tuned weight. The final identity blend reintroduces coordinate dependence, and the weights are cross-validated rather than set analytically per class. We include it as a baseline with a matched tuning budget. The contribution is analytic per-class weights, the nonlinear and robust invariant variants, and the transferability setting, where there is no pooled class structure to lean on.

Why not just keep the diagonal or coupled-diagonal model, which already works? Linderman et al.'s own result that class covariances are heterogeneous suggests that off-diagonal differences between classes carry signal the diagonal model throws away. If our full-structure estimator does not beat 80.98% AUROC, that is evidence the diagonal restriction loses little. We would state that plainly.

Why not just whiten with the small sample itself, as Ostrovskii and Rudi do? When $n_k<p$, self-whitening is undefined on the missing directions, which is exactly the obstruction proved above. Their guarantees also rely on sample splitting that small classes cannot afford.

Proposed Experiments

The decisive OOD experiment runs every estimator on ViT-B-16 features for SSB Hard and the other near-OOD and far-OOD splits used by Linderman et al. We subsample the per-class training features to 10%, 30% and 100% to move the ratio $p/n_k$ from above one to about 0.6.

The perturbation test applies a random invertible linear map with condition number from $10$ to $10^4$ to all features. Invariant methods must return identical AUROC. Identity-target and diagonal methods will drift, and the size of that drift measures how much invariance matters.

The transferability experiment recomputes shrunk H-scores and collapse indices at target sizes from 5 to 100 samples per class. It reports rank correlation with fine-tuned accuracy and stability across resamples.

A spectral diagnostic tests the prediction from the analytic-shrinkage paradox work. Chosen shrinkage intensities should stay away from zero after $R$-whitening even when the raw spectrum is spiked. To match the file's scope, we run a heavy-tail check by injecting Student-t noise and comparing the robust invariant variant with Chen et al.'s robust shrinkage.

The proposal fails if the $R$ target gives no gain over $\mathrm{diag}(R)$ on unperturbed features and no gain over coupled-diagonal on SSB Hard. In that case invariance would be a property worth having but not a source of accuracy.