Balancing Treatment Groups Along the Hierarchy: Hyperbolic Transport for Counterfactual Regression

To estimate what a treatment does for an individual, we need the outcome under the treatment they did not get, and that outcome is never observed. Neural estimators handle this by learning a representation of each patient in which treated and untreated patients look alike. They then predict the missing outcome from people on the other side. How "alike" is measured decides which untreated patients stand in for which treated ones. In medicine, patients are often described by codes from a tree-shaped ontology: diagnoses nested under disease families, drugs nested under drug classes. A flat Euclidean distance on these codes treats "two different subtypes of heart failure" and "heart failure versus a fracture" as similarly far apart. We propose to measure the gap between treated and control groups with optimal transport, the cheapest way to move one distribution of points onto another. The transport is computed in hyperbolic space, a curved geometry whose volume grows exponentially with radius, just as the number of nodes in a tree grows with depth. There, the cost of moving a patient is roughly the tree distance through the pair's closest common ancestor.

In two sentences: the balancing penalty in counterfactual regression implicitly pairs each treated patient with control patients, and its cost function decides whether those pairings respect the disease hierarchy. We make that cost the hyperbolic geodesic distance. The penalty then aligns the groups across branches of the ontology but tolerates differences within a branch. We predict this reduces the known tendency of balancing to erase prognostic information, and gives the largest gains when confounding acts at coarse levels of the tree.

Core Claim

Counterfactual regression (CFR), introduced in Learning Representations for Counterfactual Inference and Estimating individual treatment effect: generalization bounds and algorithms, bounds the error of individual effect estimates by two terms. The first is the error on observed outcomes. The second is a distance between the treated and control representation distributions. When that distance is a Wasserstein distance, the bound is only meaningful if the loss on outcomes changes smoothly (is Lipschitz) with respect to the metric used to compute the transport. The geometry of the balancing cost is therefore a modeling assumption about how outcomes vary across patients, not an implementation detail.

Our claim is that when covariates are ontology codes, outcomes vary smoothly along the tree rather than along flat directions. Outcomes change a little between siblings and a lot between distant branches. The right cost is then a tree-like geodesic. We learn a representation $\Phi$ into the Lorentz model of hyperbolic space. This model is the upper sheet of a hyperboloid, which Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometry showed trains more stably than the Poincaré ball. We then minimize

$$\mathcal{L} = \mathcal{L}_{\text{factual}} + \alpha \, W_{d_{\mathbb{H}}}\big(P(\Phi(X)\mid T=1),\, P(\Phi(X)\mid T=0)\big), \qquad d_{\mathbb{H}}(u,v) = \operatorname{arcosh}\big(-\langle u,v\rangle_{\mathcal{L}}\big).$$

Here $\mathcal{L}_{\text{factual}}$ is the error on observed outcomes and $\alpha$ is the balancing weight. $W_{d_{\mathbb{H}}}$ is an entropy-regularized (Sinkhorn) Wasserstein distance whose ground cost is the hyperbolic distance $d_{\mathbb{H}}$. $\langle\cdot,\cdot\rangle_{\mathcal{L}}$ is the Lorentzian inner product. Two-headed outcome networks, as in TARNet/CFR, read the representation after a logarithmic map to the tangent space.

The claim makes three falsifiable predictions. First, the gain over Euclidean-cost balancing should grow with the depth gap between where treatment selection acts and where outcome heterogeneity lives. Second, the gain should vanish on flat, non-hierarchical covariates. Third, the gain should come mostly from the transport cost, not from the hyperbolic encoder.

Relation to Prior Work

The closest work is Treatment-Aware Hyperbolic Representation Learning for Causal Effect Estimation with Social Networks (TAHyper). It embeds a social graph among units with a hyperbolic graph network as a proxy for hidden confounders. Its balancing, however, remains a standard Wasserstein term that is not built on hyperbolic distances. Our proposal differs on both axes: there is no graph between units, since hierarchy lives inside each unit's covariates, and the object we change is the balancing cost itself.

Unsupervised Hierarchy Matching with Optimal Transport over Hyperbolic Spaces supplies the method. It computes Sinkhorn divergences with Poincaré geodesic cost to align taxonomies, reaching 88.8% Precision@10 on English–Spanish WordNet against 1.4% for the best Euclidean baseline. It never uses transport to balance treatment groups. Its warning about branch-permutation invariance matters less for us, because both treatment groups share one encoder and therefore one coordinate system.

Representation Learning for Treatment Effect Estimation from Observational Data (SITE) motivates the proposal: global balancing can collapse useful local neighborhoods. SITE repairs this with Euclidean hard triplets. We ask instead whether choosing the right geometry lets the transport plan itself preserve neighborhoods.

Spiked-CFR: Causal Representation Learning from LLMs via Wasserstein Projection Pursuit constrains the design in two ways. It shows empirical Wasserstein balancing degrades in high dimension, at rate $n^{-1/D}$, and it fixes this with low-dimensional linear projections. This pushes us toward low-dimensional hyperbolic representations, which Poincaré Embeddings for Learning Hierarchical Representations showed can hold large trees in very few dimensions. It also makes Spiked-CFR a baseline that any low-dimension story must beat.

Geodesic Causal Inference handles outcomes in curved spaces. That is orthogonal to our setting, where the outcome is scalar and the covariates are curved.

CURE: A Pre-training Framework on Large-scale Patient Data for Treatment Effect Estimation shows claims histories carry exploitable structure. It supplies both a real-data setting and a Euclidean pre-trained competitor.

Research Program

The work has three parts.

The first is the estimator. It has three components: a Lorentz-model encoder over multi-hot ontology codes plus flat covariates such as age and labs, a hyperbolic Sinkhorn balancing term, and TARNet-style outcome heads. We add a learnable curvature, clipping of the time-like coordinate, and Riemannian Adam.

The second is a short theoretical note. It restates the CFR bound with a Wasserstein-1 distance under $d_{\mathbb{H}}$. It makes explicit that the bound tightens exactly when outcome functions are Lipschitz along tree geodesics with a smaller constant than along Euclidean directions. It also works out a toy case: a two-level tree with confounding at the root split and outcome variation at the leaves. In that case, Euclidean balancing provably shrinks leaf-level variation more than geodesic balancing does. This is the formal version of the "over-balancing" concern.

The third is a semi-synthetic benchmark family. It fixes the gap in the file, which has no public treatment-effect benchmark with real hierarchical covariates and known counterfactuals. We take real ICD and drug code structures and code frequencies from a claims cohort in the style of the MarketScan CCAE coronary artery disease cohort used by CURE, or a public ontology with sampled patients if access is limited. We then simulate treatment assignment and potential outcomes. Two knobs are controlled: the tree depth at which assignment depends on codes (the selection depth), and the depth at which the treatment effect varies (the heterogeneity depth). A third knob controls overlap sparsity at the leaves.

Baselines

The naive reference is TARNet with no balancing term, trained on the same inputs. It shows whether any balancing helps at all.

The strongest existing method is CFR with Euclidean Wasserstein balancing Estimating individual treatment effect: generalization bounds and algorithms. The file gives no numeric PEHE for it (no reported result found on Lacuna for CFR PEHE on IHDP), only that it outperformed BART and Causal Forests. Because this proposal changes only the cost of that term, CFR-Wasserstein is the direct competitor. We run it with ancestor-expanded inputs, so it has the same hierarchy information. It gets the same encoder width, the same representation dimension grid (including low dimensions), and the same Sinkhorn solver with only the cost swapped.

SITE Representation Learning for Treatment Effect Estimation from Observational Data is included because it targets the same over-balancing failure by a different route. The file reports no numbers for it (no reported result found on Lacuna for SITE numeric PEHE).

Spiked-CFR Spiked-CFR: Causal Representation Learning from LLMs via Wasserstein Projection Pursuit is included because it is the Euclidean answer to "balance in low dimension." Its projection dimension is matched to our hyperbolic dimension.

On real claims data, a CURE-style pre-trained Euclidean encoder with CFR heads is the practical competitor. CURE reports a 15.7% absolute improvement in IF-PEHE over its eight baselines on the CCAE cohort, which is the only reported causal-precision figure in this setting. On BlogCatalog and Flickr, TAHyper is the reference; it reports over 80% lower ATE error than the next best baseline on BlogCatalog, with no table values found.

We also include BART and Causal Forest on ancestor-expanded features, and nearest-neighbor matching under the exact ontology tree distance.

The key ablation is a two-by-two grid: Euclidean versus hyperbolic encoder, crossed with Euclidean versus hyperbolic transport cost. All four cells share identical inputs.

For fairness, every method gets the same tuning budget of trials. Hyperparameters, including $\alpha$, are selected by one shared criterion that uses no counterfactuals: held-out factual loss plus the influence-function PEHE surrogate. This follows from the validation gap that the CFR papers identify. Oracle-selected results are reported separately, as an upper reference only.

The proposal counts as beating CFR-Wasserstein only under three conditions. The hyperbolic-cost cells must reduce $\sqrt{\text{PEHE}}$ with non-overlapping confidence intervals across at least 20 simulation replicates in the regime where selection depth is shallower than heterogeneity depth. They must not lose on the flat control. And most of the gain must sit in the transport-cost factor of the grid.

Why not just…?

Why not just give Euclidean CFR the ancestor codes as extra features? This is the most serious threat. Ancestor-expanded multi-hot inputs hand the network the hierarchy for free, and a deep Euclidean encoder may learn tree-like features from them. The answer is empirical, and the design is built to test it. Every baseline receives ancestor-expanded inputs, so the only remaining difference is the geometry of the balancing cost. If Euclidean CFR on these inputs matches us across the depth sweep, the core claim is wrong. We would then report that the hierarchy's value lies in the inputs, not in the balancing geometry.

Why not just feed pre-trained Poincaré embeddings of the ontology into standard CFR? This changes only the input geometry and keeps a flat cost. It is exactly the "hyperbolic encoder, Euclidean cost" cell of the ablation grid. The toy analysis predicts it will still over-balance, because the flat cost in the tangent space distorts distances between far-apart leaves. The grid will show whether that prediction holds.

Why not just do optimal transport or matching directly on the known ontology tree? Tree distance on raw codes is exact and cheap, and it is a strong estimator when all confounding lives in the codes. We include it as a baseline. It cannot combine codes with continuous covariates in a learned space, and it cannot learn that some branches are irrelevant to outcomes. We expect it to fail when heterogeneity depends on labs or age that interact with codes. If it matches us even when codes are the only confounders, that limits our claim to mixed-covariate settings.

Why not just use Causal Forests or BART? Really Doing Great at Estimating CATE? A Critical Look at ML Benchmarking Practices in Treatment Effect Estimation found that neural indirect estimators beat forests on IHDP unless effects are additive. TAHyper reports BART was clearly outperformed on its graph benchmarks. We still run both on ancestor features, including a simulation arm with additive effects where forests should win.

Proposed Experiments

The main experiment is the semi-synthetic depth sweep. We vary selection depth, heterogeneity depth, and leaf overlap, and report $\sqrt{\text{PEHE}}$ and ATE error for every baseline and ablation cell. The decisive pattern is a gain that grows as selection moves toward the root while heterogeneity stays at the leaves, and disappears when the two coincide.

The second check is a negative control on IHDP, which has 25 flat covariates, and on Twins. There the hyperbolic cost should give no benefit. A gain on IHDP would suggest the improvement comes from regularization side effects rather than geometry.

The third check is a dimension sweep, from 2 to 128 dimensions, for Euclidean-cost, Spiked-CFR, and hyperbolic-cost balancing. It tests whether low-dimensional hyperbolic representations avoid the high-dimension degradation that Spiked-CFR documents while keeping outcome accuracy.

The fourth check is on the real claims cohort, where true effects are unknown. We report IF-PEHE and policy risk for the four drug pairs that CURE modeled after randomized trials. We also check whether estimated average effects move toward the published trial directions.

Finally, on BlogCatalog and Flickr we swap TAHyper's balancing cost for the hyperbolic cost, keeping its graph encoder. This tests whether the change transfers to the one existing hyperbolic causal model.