Classifiers for political text, such as stance detectors, ideology taggers and comment moderators, are trained as a single pooled model. That design is wrong whenever the same words mean different things coming from different sides. A tweet "against Biden" written by a progressive and one written by a conservative share a label but not a vocabulary. A phrase that predicts one label for one faction can predict the opposite label for another. A single model trained to minimize average error will average these opposing signals toward zero and fit the majority faction.
This note proposes a fix that needs no ideology labels. First, recover each author's position on a low-dimensional ideological map from their interaction network and their other posts, never the post being classified. Then grow a small tree of specialized classifiers over that map. A split into separate models is kept only if a held-out statistical test shows that every resulting group prefers its own model to the pooled one and to every other group's model. The bet is that unsupervised ideology recovery can stand in for the demographic group labels that subgroup methods normally require, and that it is accurate enough for that purpose. Recovery is already accurate: text-based ideal points correlate 0.94 with roll-call positions. The preference test is what keeps the method from hurting groups when the latent assignment is wrong.
Core Claim
Latent ideological coordinates are a useful grouping variable for supervised political NLP, but only in a specific way. They help when the mapping from text to label actually changes across the ideological map. They do not help merely because the population is polarized. A decoupling procedure that tests group preferences can find these regions automatically, without demographic or party labels. It should match pooled models where the mapping is shared, and beat them on cross-pressured authors where it is not.
The novelty is the pairing of two pieces that have not been combined. The first is unsupervised recovery of continuous ideology. The second is decoupling with preference guarantees, which so far has only been run on known categorical attributes in tabular data. The note also commits to a falsifiable prediction about where the gains will appear.
Prior Work and Where This Sits
Three lines of work motivate the latent-coordinate half, and we reuse their methods.
- Text-Based Ideal Points (TBIP) places each author on a continuous spectrum by letting ideology bend word frequencies within topics. It reached 0.94 correlation with vote-based ideal points on 114th Senate tweets.
- Unsupervised Detection of Contextualized Embedding Bias with Application to Ideology uses links between Reddit communities to concentrate ideology into a 6–20 dimensional subspace of BERT embeddings.
- Unsupervised User Stance Detection on Twitter and Unveiling polarization in social networks: A matrix factorization approach show that factorizing retweet and interaction matrices separates opposing camps.
All of these stop at description: they measure cluster quality or track history. None asks whether the recovered positions improve a downstream predictor. That question is ours.
The Hofmann et al. paper also constrains the design. Its finding that the main U.S. axis rotated after 2016 from left–right to pro- versus anti-Trump means a single fixed axis is unsafe. We therefore allow the tree to split on several latent dimensions, and we refit the map per time window. Ideology Takes Multiple Looks likewise shows that ideology spans several facets and that fine-tuned models reach only about 42% macro-F1 on its 12-facet task, which argues against forcing a single axis.
The decoupling half is supplied by Fairness without Harm: Decoupled Classifiers with Preference Guarantees. Its tree algorithm splits a population into groups with separate models and prunes splits using McNemar tests on held-out data. It demands two properties:
- rationality: each group prefers its own model to the pooled one;
- envy-freeness: no group prefers another group's model.
It reported up to a 30.8% accuracy gain for a subgroup on the apnea dataset. It works on observed categorical attributes in tabular data. We replace those attributes with latent ideological coordinates in text, and we must handle the uncertainty in assigning authors to groups.
The closest prior bridge is Subgroup Detection in Ideological Discussions. It clusters "attitude profiles" (who and what each user speaks positively or negatively about) to recover factions, reaching purity 0.66 against 0.49 for interaction-graph clustering and 0.52 for tf-idf clustering on Wikipedia talk pages. It treats faction detection as the end goal and never trains per-faction predictors. We use its datasets precisely because they have ground-truth sides.
A causal line of work, Adapting to Latent Subgroup Shifts via Concepts and Proxies and Prediction under Latent Subgroup Shifts with High-Dimensional Observations, proves when a latent subgroup can be adapted to. Both need hand-specified proxy and concept variables, and both are evaluated on synthetic data or images. We take a weaker but practical position. The interaction graph plays the role of a proxy that is roughly independent of the post's text given the author's ideology. We test empirically, without claiming identification, whether that suffices.
The main threat is For Robust Worst-Group Accuracy, Ignore Group Annotations. It shows that downsampling and upweighting collapse to plain-training performance with as little as 5% group-label noise. Latent ideology assignments will certainly be noisier than that. We address this head-on below.
Research Program
Stage one: build the ideological map without touching labels. For each author we estimate a latent position $z_a \in \mathbb{R}^d$, with $d$ between 1 and about 5, from two sources:
- a TBIP fit on the author's other posts, with the target post excluded;
- where an interaction graph exists, a factorization or graph-auto-encoder embedding of retweet, reply or community-link structure.
Excluding the target post is essential. Without it, the position would leak the stance label and any gain would be circular. We keep posterior uncertainty over $z_a$, not just a point estimate.
Stage two: grow a latent decoupling tree. We start from a pooled classifier, here BERTweet fine-tuned as usual. At each node, candidate splits are thresholds on one latent dimension. A candidate child model is the parent's encoder with a separate classification head, plus optional light adapter fine-tuning, trained on authors assigned to that side. Write $\mathrm{err}_g(h)$ for the held-out error of classifier $h$ on group $g$. A split into groups $g$ with models $h_g$ is kept only if, for every $g$ and every sibling $g'$,
$$ \mathrm{err}_g(h_g) \le \mathrm{err}_g(h_{\text{parent}}) \quad\text{and}\quad \mathrm{err}_g(h_g) \le \mathrm{err}_g(h_{g'}), $$
with each inequality checked by a McNemar test on a held-out split disjoint by author. These are the rationality and envy-freeness conditions of the decoupling paper, now evaluated on latent rather than observed groups.
Two changes target noisy group assignment.
- Soft training. Each author's training weight in a child equals the posterior probability of falling on that side. Authors near a boundary therefore contribute to both children instead of being hard-assigned wrongly.
- Soft prediction. At test time, predictions are averaged over the author's position posterior.
Stage three: predict where gains occur and check the prediction. For each node we compute a heterogeneity score: the held-out gain of a pooled model with explicit text-by-position interaction terms over a pooled model without them. Our claim predicts two things. Accepted splits should coincide with high heterogeneity. Gains should concentrate on cross-pressured authors, such as left-wing critics of Biden in P-Stance, or users whose MITweet facets disagree with their dominant axis. If gains are instead spread evenly or appear at random splits, the mechanism is not what we claim.
Baselines
Pooled model (reference point). The do-nothing reference is pooled BERTweet fine-tuned per target on P-Stance, which reports 80.53% macro-F1 in-target. The strongest reported number on this benchmark is BERTweet with uncertainty-aware self-training on the 2.8M-tweet unlabeled pool, at 82.42% macro-F1. That system uses the same unlabeled pool from which we derive author positions, so it is the fair strongest competitor. We will rerun it with our author-disjoint splits, since author-disjoint numbers will differ from the reported ones, and compare all methods on those splits. Beating it means higher overall macro-F1 together with higher worst-latent-group macro-F1, with no group significantly worse.
Pooled model conditioned on position. This is the most dangerous baseline: the same pooled model with $z_a$ concatenated to the text representation, a modern version of the conditioning in Demographic Factors Improve Classification Performance. That paper found gains of only 0.2–1.0 F1, significant in 8 of 30 settings. A large encoder given the coordinate can in principle learn the same interactions that decoupling exploits. This baseline gets the same encoder, the same position estimates and the same tuning budget.
Pseudo-group upweighting (RAD). This is RAD-UW from the Stromberg et al. paper. It marks examples misclassified by a heavily regularized linear probe as pseudo-minority and upweights them. It is the strongest annotation-free subgroup method in the file. It runs on identical frozen features with matched hyperparameter search.
Group DRO on latent groups. Group DRO is worst-group robust training, which upweights whichever group currently has the highest loss. We train it on hard-thresholded latent groups. This isolates whether our gains come from decoupling and preference testing or simply from having latent groups at all.
Faction-detection groups. On Createdebate and Politicalforum we additionally decouple on clusters from attitude-profile clustering, and on the gold debate sides. The gold-side run is an oracle upper bound.
Ablations. Three ablations isolate the key ingredients:
- remove the preference test, so all splits up to a fixed depth are kept;
- replace soft posterior weighting with hard assignment;
- replace graph- and other-post positions with a position inferred from the target post itself, to expose leakage.
All methods share the backbone, the compute and a 30-trial hyperparameter budget.
Why not just…?
Why not just give the pooled model the ideology coordinate as an input feature? This is a real threat. If the conditioned pooled model matches the tree everywhere, the honest conclusion is that the contribution lies in the latent coordinate, not in decoupling. We would report exactly that. Decoupling should win in two situations. One is when cross-pressured regions are small, so a shared model's capacity is dominated by the majority; Negriff et al. and Su et al. show that inverted feature effects are what pooling averages away. The other is when a guarantee is required, because conditioning gives no assurance that any faction is not made worse off. The head-to-head comparison and the per-group McNemar results settle the question.
Why not just use Group DRO or upweighting on the inferred ideology groups? Stromberg et al. show that these methods lose their advantage at 5% group-label noise. Latent assignments are far noisier than that. Group DRO also optimizes the worst group, possibly at a cost to others, whereas the preference test forbids any group losing. The Group DRO on latent groups baseline measures this directly. On Createdebate we can inject controlled assignment noise by swapping gold sides at 5%, 15% and 30% and watch which method degrades first.
Why not just fine-tune a bigger pooled model? Pooled fine-tuning reaches only about 42% macro-F1 on MITweet's facets and 56.0% worst-group accuracy on CivilComments-WILDS (WILDS). Scale does not obviously fix heterogeneity in what labels mean. Still, we include a larger encoder in the pooled baseline. If its gain over the smaller encoder exceeds the tree's gain over the smaller encoder, the method is not worth its complexity.
Why not just cluster the text with k-means and decouple on that? Text clustering reached only 0.52 purity against 0.66 for attitude profiles on Wikipedia talk pages. Shared vocabulary across sides makes such clusters track topic rather than faction. We run this cheap variant anyway as a further grouping choice.
Proposed Experiments
Mechanism with ground truth. On Createdebate and Politicalforum, sides are known. We first measure how well latent positions recover the sides, by purity against the reported attitude-profile results; for Politicalforum the reported attitude-profile result is purity 0.64 and entropy 0.69. We then train post-level stance or agreement classifiers decoupled on three kinds of groups: latent groups, attitude-profile clusters and gold sides. The decisive check is whether latent-group decoupling recovers a large share of the oracle gold-side gain, and how that share falls as group-recovery purity falls.
Main benchmark. P-Stance, with author positions derived from the unlabeled pool where author identifiers and interactions can be recovered. Otherwise positions come from TBIP on each author's other tweets, and we report which authors have which kind of position. Metrics are overall macro-F1, worst-latent-group macro-F1, number of preference violations on the test set, and the per-region gain plotted against the heterogeneity score.
Facets and drift. On MITweet we test whether accepted splits use different latent dimensions for different facets. On Reddit Politosphere windows before and after 2016 we test whether a tree learned before 2016 violates its preference guarantees after 2016, and whether refitting the map restores them. This directly probes the axis rotation reported by Hofmann et al.
Fairness audit. On the YouTube moderation data from Reasoning about Political Bias in Content Moderation, we replace channel-level ideology with latent commenter positions. We ask whether any latent faction prefers another faction's moderation model, which is a finer audit than the channel-level comparison in that paper. As an implementation check for the worst-group code, we first reproduce the WILDS CivilComments numbers of 56.0% for pooled training and 70.0% for Group DRO.
What would falsify the idea. The claim fails if, on author-disjoint P-Stance, the conditioned pooled model or self-training matches the tree on both overall and worst-group macro-F1. It also fails if the tree's accepted splits do not line up with the heterogeneity score. Either result would show that latent ideology is useful as a feature but that subgroup-specific modeling on top of it adds nothing.
References
- Text-Based Ideal Points
- Unsupervised Detection of Contextualized Embedding Bias with Application to Ideology
- Unsupervised User Stance Detection on Twitter
- Unveiling polarization in social networks: A matrix factorization approach
- Ideology Takes Multiple Looks: A High-Quality Dataset for Multifaceted Ideology Detection
- Fairness without Harm: Decoupled Classifiers with Preference Guarantees
- Subgroup Detection in Ideological Discussions
- Adapting to Latent Subgroup Shifts via Concepts and Proxies
- Prediction under Latent Subgroup Shifts with High-dimensional Observations
- For Robust Worst-Group Accuracy, Ignore Group Annotations
- P-Stance: A Large Dataset for Stance Detection in Political Domain
- Demographic Factors Improve Classification Performance
- WILDS: A Benchmark of in-the-Wild Distribution Shifts
- Reasoning about Political Bias in Content Moderation