Which Variables Go Extreme Together, and When: Covariate-Dependent Sparse Faces of Multivariate Extremes

In large monitoring networks such as air-quality stations, river gauges, or regional power grids, a crisis rarely hits every sensor at once. Usually a small group of variables spikes together while the rest stay normal. Extreme value theory (the statistics of rare, very large values) has tools that find these groups. They do it by looking only at observations that are extreme overall and recording which coordinates are large in each of them. Two such tools are DAMEX, which partitions the space into $2^d-1$ "cones," one per subset of variables that are jointly large (Goix et al.), and MEXICO, which clusters features that spike together (Jalalzai et al.). Both learn one fixed answer for all conditions. In practice, which stations pollute together plausibly depends on wind direction, temperature, and season, and which grid regions hit extreme net-load together plausibly depends on the weather forecast.

We propose a model in which the set of jointly extreme variables (a "face" of the extremes) is drawn from a small learned dictionary of faces, with probabilities that change with exogenous covariates. The central methodological point is that margins must be standardized conditionally on the covariates before faces are counted. Otherwise a covariate that only makes one station dirtier on average will look as if it changes which stations are extreme together. Bodik, Mhalla and Chavez-Demoulin identified this trap and solved it for pairs of variables only. Our claim is that it can be solved in $d$ dimensions by working with sparse faces instead of full dependence functions. The deliverable is a method that answers, for a network of 15–30 variables, which small groups go extreme together, under which conditions, and which covariates drive the joint behavior rather than the individual levels.

Core Claim

Fix covariates $x$, such as wind, temperature, NWP fields, or calendar. Transform each variable $j$ to a unit-Pareto scale using its conditional distribution $F_j(\cdot \mid x)$. Let $R$ be the largest transformed coordinate and $\Theta$ the vector of transformed values divided by $R$. Among exceedances $R>t$, the face $S(\Theta)=\{j:\Theta_j>\epsilon\}$ records which variables are concomitantly large. We model

$$P\big(S = S_k \mid R>t,\, x\big) = \frac{\exp(a_k + b_k^\top \phi(x))}{\sum_{l\in\mathcal{D}} \exp(a_l + b_l^\top \phi(x))},\qquad S_k\in\mathcal{D},$$

where $\mathcal{D}$ is a small learned dictionary of faces, $a_k$ is the baseline log-weight of face $k$, $\phi(x)$ is a spline expansion of the covariates, and $b_k$ holds the coefficients for face $k$. A group-lasso penalty, applied per covariate across all $b_k$, removes covariates that do not affect the dependence structure. Faces may overlap, so a station can belong to a summer ozone face and a winter $\text{NO}_x$ face. This removes MEXICO's disjoint-cluster restriction. Face membership is soft: each exceedance is assigned to dictionary faces through an EM step, so the hard tolerance $\epsilon$ in DAMEX becomes a likelihood weighting rather than a knife edge. The dictionary itself is selected by pruning faces whose fitted mass stays below a floor at every covariate value. This replaces DAMEX's single global minimal-mass rule $\mu_{\min}$.

Relation to Prior Work

The supporting literature plays three roles. One group motivates the gap. DAMEX (Goix et al., extended in Goix et al.) and MEXICO (Jalalzai et al.) show that extremal mass in real data concentrates on few low-dimensional faces. Goix et al. report 99.6% of the mass on faces of dimension at most 20 in a 50-dimensional wave dataset. Both methods estimate these faces unconditionally. Spectral clustering of extreme angles (Avella Medina et al.) and tree-mixture model averaging (Bayesian model averaging over tree-based dependence structures) likewise learn static structure. We keep their sparse-support view and add covariate dependence.

A second group constrains the design. The bivariate LogMax/invariant-causal-prediction work (Bodik et al.) shows that geographic distance drives joint UK $\text{NO}_2$ extremes, while site type only moves margins. This tells us that naive conditioning conflates the two, and it is why conditional margins are built into our model. The same authors flag scaling beyond $d=2$ as open. Our route avoids modeling a $d$-dimensional extremal coefficient by modeling a categorical distribution over a sparse face set. The Spanish $\text{PM}_{10}$ model (Wang et al.) finds covariates acting in opposite directions on moderate and extreme levels. That finding warns against reusing bulk-level covariate effects in the tail, so our face model is fit only on exceedances.

A third group supplies technique. The net-load paper (Probabilistic Forecasting of Regional Net-Load) supplies the conditional generalized Pareto margins with weather-dependent scale and shape. It does this for each of 14 regions separately and leaves joint tails open; our method models exactly that joint part. SEPaLS (Arbel et al.) also selects sparse exogenous drivers of extremes, but for a single response and one projection direction. Our group penalty plays the analogous selection role for a vector of responses.

Research Program

The work has three parts. The first is estimation. We fit conditional margins by gradient-boosted quantile regression in the body and a covariate-dependent generalized Pareto tail above a high quantile. We then extract exceedances and fit the face model by penalized EM. Hyperparameters ($t$, the spline basis, the penalty) are chosen by held-out conditional log-likelihood of face labels. This gives a data-driven answer to the tuning burden that both DAMEX and MEXICO leave manual.

The second part is analysis. The goal is a support-recovery result in the spirit of MEXICO's $O_P(1/\sqrt{k})$ excess-risk bound, where $k$ is the number of exceedances. We will state it under smoothness of face probabilities in $x$ and a minimum face-mass condition, and we expect the rate to degrade with covariate dimension through the spline basis. We will also characterize when unconditional standardization provably creates spurious covariate effects on faces. This happens when a covariate shifts one margin's tail scale, which changes how often that coordinate crosses $\epsilon$.

The third part is a diagnostic. To separate dependence drivers from margin drivers, each selected covariate is re-tested by refitting with that covariate removed only from the margins. A covariate that matters only through margins should then appear in $b$ spuriously. That pattern is the signature we want the full method to avoid.

Baselines

The naive reference point is unconditional DAMEX. It estimates face masses once from all exceedances under rank-standardized margins and predicts the same face distribution for every covariate value. Any gain over it is the value of conditioning.

The strongest competitor is stratified DAMEX. Covariates are binned (wind sector by season, or NWP-quantile bins), and DAMEX is run separately in each bin with margins standardized within bin. This is the obvious practitioner solution and it already conditions margins. It will receive the same exceedance threshold and the same tuning budget as our method, with bin edges chosen by the same held-out likelihood. Beating it means higher held-out conditional log-likelihood and better face-recovery F1 once covariates are continuous or exceed two dimensions, where bins starve.

A second strong competitor is a gradient-boosted multinomial classifier trained to predict each exceedance's hard DAMEX face label from $x$, again on conditionally standardized margins. It isolates whether our dictionary learning and soft assignment add anything beyond off-the-shelf classification.

Two further references complete the comparison. MEXICO is the static support-recovery reference. On the synthetic logistic benchmark at $p=200$, Jalalzai et al. report a v-Measure of 0.972, against 0.951 for spherical k-means. Our method with constant covariates should match this. All-pairs LogMax regression (Bodik et al.) is the pairwise conditional reference. Its pairwise extremal coefficients will be thresholded into groups for comparison.

The key ablation is our full model with unconditional rank margins. If this ablation selects margin-only decoy covariates, it confirms the central claim. No reported result was found on Lacuna for covariate-conditional face estimation on any dataset in the file, so all conditional comparisons are new runs.

Why not just…?

Why not just bin the covariates and run DAMEX in each bin? This is the real threat. With one discrete covariate and many exceedances it may match our method, and we will report that honestly. Exceedances are scarce by construction: $k$ is a small fraction of $n$, and binning divides it again by the number of bins. Face masses in each bin then become noisy, and faces seen in one bin are missed in another. Our synthetic experiments vary covariate dimension and $k$ to locate the crossover. If stratification wins everywhere, we will reframe the contribution as the conditional-margin diagnostic alone.

Why not just train a classifier from covariates to DAMEX face labels? It is a subset of our model, and it is included as a baseline. Its weaknesses are expected to be label noise from the hard $\epsilon$ boundary and the lack of any mechanism to share mass between near-identical faces among up to $2^d-1$ classes. The comparison will show whether soft assignment and dictionary pruning matter. If they do not, the classifier on conditional margins is the method, and that is still a useful finding.

Why not just feed covariates into a deep forecaster with a quantile loss? Such models predict marginal levels, not which groups co-exceed. The evidence also suggests they are weak in the tail: Air Quality Arena reports that in volatile Indian networks, top supervised models and foundation models struggled to beat a seasonal-naive baseline. Hasan et al. show that unconstrained neural estimators break validity constraints of tail dependence. We will nonetheless include a PatchTST-style model with covariates for joint-exceedance probability estimation on the California data.

Why not just run the bivariate causal method on all pairs? Pairwise extremal coefficients do not determine which triples or larger groups co-exceed. They also scale as $d^2$ separate fits without any shared sparsity. We will test this directly through face F1 on synthetic faces of size three and above.

Proposed Experiments

Synthetic data come from asymmetric logistic models with 20–200 variables. Face weights vary smoothly with three to ten covariates, of which two are true dependence drivers and one is a decoy that shifts only one margin's tail scale. We measure face-recovery F1, covariate true and false selection, and held-out conditional log-likelihood. The decisive check is that the full method excludes the decoy while the unconditional-margin ablation includes it.

On real data, the primary set is the California EPA AQS pollution-and-weather data (daily CO, NO, $\text{NO}_2$, $\text{O}_3$ and four weather variables at 21 sites) used by the tree-averaging paper, which reports cluster recovery above 80% for its static model. The second is the GB 14-region net-load data with ECMWF forecasts from the net-load paper. There, the conditional margins reduced upward reserve by up to 24.6% versus a static generalized Pareto model. We test whether modeling joint regional co-exceedance further sharpens system-level reserve estimates, scored by calibration of predicted joint-exceedance probabilities on held-out years. The UK $\text{NO}_2$ data from Bodik et al. serve as a consistency check: our method should recover distance, not site type, as a dependence driver.

For a covariate-free sanity check, the Small Flows intrusion benchmark tests whether our scorer, reduced to constant covariates, stays near DAMEX's reported ROC-AUC of 0.710. MEXICO reaches 0.892 there. The method succeeds if it beats stratified DAMEX and the classifier on held-out conditional likelihood and face F1 in the continuous-covariate regimes, and if it avoids decoy selection where the ablation fails.