Factor-Specific Transfer of Causal Representations over Hyperbolic Task Trees

When a treatment-effect model is trained on many related populations, such as hospitals, tumor sites, or user communities, the populations rarely differ in one uniform way. The biology linking a patient's state to their outcome is often shared widely. The rules by which doctors at a particular site decide who gets treated are local. Current transfer methods share or separate whole representations at once. We propose to split each population's representation by causal role and let each piece be shared over a different distance in a learned tree of populations. The pieces are the variables that drive both treatment and outcome (confounders), the variables that drive only treatment (instruments), the variables that drive only the outcome (risk factors), and pure noise. The tree is embedded in hyperbolic space, a curved geometry in which trees fit with little distortion.

The core bet is that "how far knowledge travels" is a property of a causal factor, not of a task. Outcome mechanisms should be pooled across distant branches. Treatment-assignment mechanisms should be pooled only among close siblings. Instruments should barely be pooled at all. If this holds, a model that gets the per-factor sharing range right should estimate individual treatment effects better in small target populations than models that pool everything, share nothing, or share by data availability. The learned ranges are also scientifically readable: they say which parts of a causal problem generalize.

What exists and what is missing

Three bodies of work each cover two of the three ingredients. Disentangled treatment-effect estimators split covariates into causal roles within a single population. TEDVAE routes three latent blocks to separate treatment and outcome heads. MIM-DRCFR adds mutual-information penalties between blocks. DRI-ITE adds a dedicated noise block with a reconstruction and orthogonality loss. Selective-weighting methods (DPSW, DTRNet) balance treated and control groups only on the confounder block. These papers supply our per-population estimator. None of them transfers across populations.

Causal transfer methods move knowledge between populations without separating causal roles. HTCE-learners split features into shared and private encoders according to which features each domain observes. ℓ1-TCL fine-tunes source parameters under a sparse-shift penalty. MTAL treats tumor sites in basket trials as tasks with one adversarially balanced shared generator. These methods motivate the gap: they treat a population's difference from its neighbors as one undifferentiated quantity.

CORE-MTL is the closest structural precedent. It shows that disentangling a semantic stream from a nuisance stream removes gradient conflict between tasks. However, it uses only two streams, does no treatment-effect estimation, and uses no task geometry.

On the geometry side, HVAE learns domain-specific curvature for cross-domain recommendation. HyperKT propagates knowledge over a known WordNet tree in the Poincaré ball. TAHyper embeds a social network hyperbolically as a confounder proxy, but it balances the whole embedding globally. That is exactly the pitfall the disentanglement papers warn about.

No paper in this corpus combines all three ingredients. More importantly, none asks whether different causal factors should be shared over different ranges. That question is the contribution. Hyperbolic geometry is the representation of task relatedness, not the point in itself.

Core Claim

Given $K$ related causal tasks, each task is fit with a DRI-ITE-style encoder producing four blocks: confounders $Z_C$, instruments $Z_I$, risk factors $Z_A$, and noise $Z_N$. Each block has a task-specific head with parameters $\theta_{f,k}$, for factor $f$ and task $k$. Each task also has a position $u_k$ on the Lorentz model of hyperbolic space. Heads of the same factor are pulled together across tasks by a penalty that decays with hyperbolic distance at a factor-specific rate:

$$\mathcal{R} = \sum_{f \in \{C,I,A\}} \lambda_f \sum_{k \neq j} \exp\!\left(-\frac{d_H(u_k,u_j)}{\tau_f}\right) \|\theta_{f,k}-\theta_{f,j}\|^2 .$$

Here $d_H$ is geodesic distance on the hyperboloid, $\lambda_f$ is the overall sharing strength for factor $f$, and $\tau_f$ is that factor's transfer range.

We claim three things. First, on task families where outcome mechanisms and assignment policies drift at different rates, the learned ranges order as $\tau_A > \tau_C > \tau_I$. Second, matching these ranges to the true drift rates lowers target-task error relative to any single shared range. Third, when the task tree is unknown, the hyperbolic task embedding recovers it well enough to match a model given the true tree.

Research Program

The estimator inside each task reuses known parts. These are the four-block encoder with full-input reconstruction and orthogonality from DRI-ITE, routed heads as in TEDVAE, and balancing applied only to $Z_C$, within each task, following DPSW. The block encoders themselves stay Euclidean and are shared across tasks. Gap 4 in the file asks whether independence penalties work in curved space, and we deliberately do not take that on. Covariates in IHDP- or Twins-style tables are not hierarchical, and curving them invites the topology mismatch noted for the Lorentz model.

Task positions cannot be learned by minimizing $\mathcal{R}$ jointly. Doing so would simply push every task apart. We therefore fit them in two stages. Stage one trains all tasks with weak uniform pooling. It then computes pairwise dissimilarities between tasks' risk-factor heads, the factor we expect to carry shared mechanism. These dissimilarities are embedded with Riemannian gradient descent in the Lorentz model, following Nickel and Kiela's method. We use a low dimension, 2 to 5, because hyperbolic space represents trees faithfully at that scale, as shown by Poincaré embeddings. Stage two freezes $u_k$, adds $\mathcal{R}$, and selects $\tau_f$ by held-out factual loss on the target task. An optional third stage re-estimates $u_k$ and repeats.

The analysis has three parts. We read off the learned $\tau_f$. We check whether task norms track specificity, meaning whether broad populations sit near the origin. We test whether embedding tasks by their propensity heads instead yields a different, shallower tree, which would itself be evidence that assignment policies cluster differently from biology.

A known risk is that the blocks are not identifiable. If confounder and risk-factor information leak into each other, per-factor ranges become meaningless. We track this on synthetic data by measuring how well each block predicts the true generating variables. We report results only where separation is verified.

Baselines

The naive reference is a target-only model: the same four-block DRI-ITE estimator trained on the target task alone. A second naive reference pools all tasks with a task indicator and uses a single set of heads. These two bracket the sharing spectrum. HTCE reports that target-only on Twins reaches a PEHE (the root-mean-square error of estimated individual effects) of 0.20.

The strongest existing transfer competitor is HTCE-DR-learner. It is the only method in the file with a reported causal-transfer gain on a standard benchmark: PEHE 0.05 on Twins versus 0.20 target-only, in HTCE. We will reproduce that setting first. Beating it there requires a PEHE below 0.05 over matched seeds, which is a high bar. On our hierarchical task families we expect the advantage to appear mainly where assignment and outcome drift differ, so on Twins we would accept parity.

We also run ℓ1-TCL as the sparse-shift competitor, MTAL as the multi-task adversarial competitor, and pooled MIM-DRCFR as the strongest single-population disentangler. For the last, the file reports square-root PEHE 0.38 on IHDP, versus 0.58 for TEDVAE. For high-dimensional noise, DRI-ITE reports a PEHE of about 1.15 to 1.23 with 20 injected irrelevant variables, against about 1.41 for baselines.

On the new hierarchical task families there is no reported result on Lacuna for any method. All baselines will be run by us with the same encoder widths, the same target data, and the same hyperparameter search budget (equal trials per method), selected by factual validation loss.

Two ablations isolate the key ingredients. The first uses a single shared range $\tau$ for all factors. Comparing it with the full model isolates factor-specific transfer. The second places task embeddings in Euclidean space with the same dimension and kernel, which isolates the geometry.

Why not just…?

Why not just use hierarchical shrinkage on a known task tree? When the tree is known, for example tumor lineage, a Euclidean tree-structured prior with per-factor shrinkage strengths captures most of our claim. This is a real threat to the hyperbolic component. We therefore include it as an oracle-tree baseline with the same per-factor ranges. The hyperbolic embedding earns its place only in two cases: when the tree is unknown or corrupted, where we degrade the given tree by random edge rewiring, or when tasks are numerous and the tree is deep. If the oracle-tree baseline wins everywhere, we will report the factor-specific result and drop the geometry claim.

Why not just pool everything into one disentangled model with a task indicator? Pooling shares the confounder and instrument blocks across sites whose assignment policies differ. Balancing then corrects the wrong selection bias. This is the global-balancing pathology TEDVAE and GDC describe, now acting across tasks. The synthetic drift experiment shows directly whether pooled MIM-DRCFR degrades as assignment drift grows while outcome drift is held fixed.

Why not just use shared/private encoders or a sparse parameter shift? HTCE separates information by which features each domain observes, not by causal role. ℓ1-TCL assumes a sparse difference, but a wholesale change in prescribing policy is dense in the propensity model and sparse in the outcome model. Both methods are run on the drift grid. If the shift is genuinely sparse and the same across factors, ℓ1-TCL should match us, and we say so.

Why not just use Euclidean task embeddings? With few tasks, a Euclidean embedding can fit any distance pattern, so the geometry should only help as tree depth and task count grow. The Euclidean-embedding ablation is run across depths 2 to 6. We expect any gap to widen with depth. If it does not, the geometry is unnecessary.

Proposed Experiments

The decisive testbed is semi-synthetic. We take IHDP covariates (25 features) and Twins covariates. We then generate task trees of depth 2 to 6 with up to several dozen leaves. Along each edge, the outcome function, the assignment function, and the instrument weights drift at independently controlled rates, and we inject irrelevant variables as in DRI-ITE. Each leaf becomes a target task with few treated units, mirroring the 58-treated sepsis target in ℓ1-TCL.

The metrics are PEHE and ATE error on target leaves, recovery of $\tau_f$ ordering against the true drift rates, and tree recovery, measured as the rank correlation between hyperbolic distances and true tree distances. A first decisive check: on configurations where the outcome model drifts slowly and the assignment model drifts fast, the full model should beat the single-$\tau$ ablation. On configurations where all three drift at the same rate, the two should tie. A gain that appears in both cases would indicate the benefit comes from something other than factor-specific sharing.

Two real-data checks follow. First, we reproduce the Twins heterogeneous-transfer setting of HTCE, where we expect parity. Second, we use BlogCatalog and Flickr, treating network communities as tasks in a transfer version of TAHyper's setting. There we additionally test whether replacing TAHyper's global Wasserstein balancing with confounder-only balancing preserves its reported ATE gains, which exceeded 80% error reduction over the next baseline on BlogCatalog. News-16, where MTAL reports ATE error 5.12 versus 5.76 for the next baseline, serves as a stress test with many treatments, treating topic clusters as tasks.

The project succeeds if factor-specific ranges are recovered in the right order and improve target PEHE exactly where the drift rates diverge. The hyperbolic component succeeds only if it beats the Euclidean embedding as trees deepen and matches the oracle tree when the tree is hidden.