When we train a machine learning model to predict a target variable $Y$ from a set of features $X$, we are modeling the conditional distribution $P(Y|X)$. In many real-world scenarios, labeled data is scarce, but unlabeled data is abundant. Traditional semi-supervised learning (SSL) relies on the assumption that the marginal distribution of the features, $P(X)$, contains structural information—such as clusters or low-dimensional manifolds—that aligns with the decision boundaries of $P(Y|X)$.
However, a fundamental asymmetry emerges when we view this problem through a causal lens. Whether unlabeled data can actually help a model depends entirely on the direction of the causal arrow between $X$ and $Y$. By partitioning features based on their causal roles, we can understand exactly when SSL is theoretically viable and design algorithms that exploit these structural boundaries.
The Causal-Anticausal Asymmetry
To build intuition, consider two different prediction tasks:
- The Causal Direction ($X \to Y$): You want to predict whether a glass will break ($Y$) based on the height from which it was dropped ($X$).
- The Anticausal Direction ($Y \to X$): You want to predict a patient's underlying disease ($Y$) based on their observed clinical symptoms ($X$).
In the anticausal direction, the target $Y$ is the generative source of the features $X$. Because the symptoms are generated by the disease, the marginal distribution $P(X)$ is a mixture of distributions dictated by $Y$. If you collect millions of unlabeled symptom profiles, you will naturally see clusters emerge in $P(X)$ that correspond to different diseases. As a result, SSL is highly effective here; the shape of $P(X)$ directly informs the boundaries of $P(Y|X)$.
In the causal direction, the situation is completely inverted. According to the Principle of Independent Causal Mechanisms (ICM), the distribution of a cause $P(X)$ and the mechanism that maps the cause to its effect $P(Y|X)$ are independent properties of nature. Collecting millions of unlabeled drop heights $P(X)$ tells you about the habits of the person dropping the glass, but it reveals absolutely nothing about the physical threshold at which glass breaks $P(Y|X)$.
As Bernhard Schölkopf demonstrates, this independence implies that semi-supervised learning is theoretically impossible for purely causal learning problems. This insight is powerful because it explains empirical failures across different domains: Zhijing Jin 0001 et al. showed that this exact directionality explains why SSL excels in anticausal NLP tasks (like sentiment analysis) but struggles in causal ones (like machine translation).
Causal Direction (SSL Fails):
[ Cause: X ] ----> [ Effect: Y ]
P(X) and P(Y|X) are independent mechanisms. Unlabeled X is uninformative.
Anticausal Direction (SSL Succeeds):
[ Cause: Y ] ----> [ Effect: X ]
P(X) is a mixture generated by Y. Unlabeled X reveals the structure of Y.
Why Blind Clustering Degrades Performance
When a dataset contains features where the cause determines the label, traditional SSL algorithms can actually perform worse than simple supervised learning. Algorithms that rely on the standard cluster assumption assume that all features are equally informative for identifying decision boundaries.
When applied to causal features, these algorithms attempt to cluster data based on variations in $P(X)$ that are completely irrelevant to the target mechanism. As Yu Yao et al. observe in the context of learning with noisy labels, treating low-confidence samples as unlabeled data in causal structures causes the model to discard highly informative points, leading to severe performance degradation. Because the feature distribution lacks information about the decision boundary, unsupervised clustering simply misguides the model.
The Conditional Cluster Assumption
Real-world datasets are rarely purely causal or purely anticausal; they typically contain a mixture of both. For example, if you are predicting a patient's health outcome, your features might include both their pre-existing genetic risks (causes) and their post-treatment lab results (effects).
If we throw all these features into a standard SSL pipeline, the uninformative marginal distribution of the causal features will dilute and corrupt the informative signal from the effect features. To solve this, Julius von Kügelgen et al. introduced the strategy of causal feature partitioning.
Instead of treating the feature space as a homogenous block, the input variables are explicitly divided into a set of causes ($X_C$) and a set of effects ($X_E$) relative to the target $Y$.
Mixed Feature Space:
[ Causes: X_C ] ----> [ Target: Y ] ----> [ Effects: X_E ]
By partitioning the features this way, the authors formulated the Conditional Cluster Assumption. This principle states that the target label $Y$ remains consistent within clusters formed by the conditional distribution of the effects given the causes:
$$P(X_E | X_C)$$
Why does this work? Conditioning on the causes ($X_C$) accounts for the independent background variations that set the stage for the causal mechanism. Once the causes are fixed, any remaining variation observed in the effects ($X_E$) must be driven by the underlying target variable $Y$.
By clustering the functional mappings from causes to effects rather than looking at spatial proximity in the raw feature space, models can successfully leverage unlabeled data even in complex, mixed-causal environments. Abigail E Moore et al. expanded on this by showing that as long as the broader causal graph includes effects or "spouses" of the label within its Markov Blanket, unlabeled data remains highly valuable for refining predictions.
Turning Asymmetry into a Diagnostic Tool
The fact that unlabeled data only benefits models when the causal arrow flows from the target back to the predictor can be turned into a powerful diagnostic tool for causal discovery.
Instead of guessing the causal direction, Nan Zhang et al. demonstrate that you can detect reverse causality by simply measuring the predictive lift provided by unlabeled data. If adding unlabeled examples of a variable significantly improves your ability to predict the target, it provides strong statistical evidence that the target is acting as the generative cause of that variable.
Similarly, in regression tasks where a hidden cause must be inferred from multiple observed effects, Shahine Bouabid et al. show that structural constraints derived from independent predictors can be estimated using only unlabeled data. This allows practitioners to project their regression models into a valid, causally consistent hypothesis subspace, drastically reducing the amount of expensive labeled data required to achieve high accuracy.
Key Foundational Work
Domain Adaptation under Target and Conditional Shift (2013) — contributed 1 concept elements to this direction.
The Randomized Causation Coefficient (2014) — contributed 1 concept elements to this direction.