Who Changed the Words? Separating Selection from Semantic Change After Deplatforming

When a platform bans a community, such as Reddit's removal of r/The_Donald, the most committed members often regroup on a fringe site, and the language they use there looks different. It may be more hostile, more coded, or full of new slang. Prior studies have read that difference as evidence that unmoderated spaces radicalize their users. That reading skips a basic question. The people posting after the move are not the same population as before. Casual members drop off and a dedicated core remains, so a word can appear to "change meaning" simply because the remaining speakers always used it differently. This proposal measures how word meanings shift across a deplatforming event and splits the observed shift into three parts. The first is selection: who chose to migrate. The second is genuine change within the same people after they move. The third is background drift that also happened in comparable communities that were never banned.

The tool is a contextual language model, BERT, whose word vectors depend on the surrounding sentence. We first continue its training on the community's own pre-ban posts so that it represents the community's slang well. For each target word, the usages of every user are grouped into discrete senses. Comparing these per-user sense distributions across the ban, for users matched by username, lets us attribute each shift to selection, within-user change, or background drift. A word whose apparent drift is mostly selection tells us something about who migrates. A word whose drift survives the within-user and control comparisons is evidence of new coded language forming after the move.

Core Claim

Semantic change measured at the community level across a migration mixes changes in speaker composition with changes in meaning. Contextual sense distributions computed per user, together with cross-platform user matching and an unbanned control community, can separate the two. We expect that a substantial share of the apparent post-ban shift in fixation and out-group vocabulary is selection. We also expect a smaller set of words to show within-user sense change that does not occur in controls, and these words should be enriched for coded usage. If both expectations hold, existing conclusions about "radicalization after migration" need re-reading. If the within-user share is near zero, that is equally informative.

Prior Work and What It Leaves Open

The motivating evidence comes from migration studies. Do Platform Migrations Compromise Content Moderation? runs a before/after design on r/The_Donald→thedonald.win and r/Incels→incels.co. It scores content with Perspective API, LIWC (a psycholinguistic word-category dictionary), and hand-built fixation dictionaries. It finds escalation for the former and not the latter, and it explicitly notes that exact-username matching samples super-users and that self-selection inflates per-user activity. Its measures cannot see meaning change in a fixed word. From Dogwhistles to Bullhorns builds a glossary of 340 dogwhistles (terms with a benign surface meaning and a hidden in-group meaning) and shows that substituting them for slurs makes Perspective scores collapse. That makes a Perspective-based migration study blind to exactly the change we target. Free speech or Free Hate Speech? shows general detectors fail on Parler and fine-tunes DistilBERT on 10,000 annotated posts. Its task is static supervised classification of a single snapshot, not change over time.

The method comes from semantic-change work. SST-BERT clusters BERT token vectors (last four layers concatenated) into senses with k-means and ranks change by Jensen-Shannon distance between sense distributions. It has been tested only on historical book and newspaper corpora. Semantic shift in social networks measures shift across 45 subreddits with diachronic static word2vec-style embeddings. It controls frequency effects with a shuffled null ("rectified change") and finds that community size, stability, and clustering only predict shift jointly. We reuse its null-distribution idea. Our case differs from both papers because the population changes abruptly at the event, and neither separates speaker composition from meaning.

Two papers constrain the design. Determining Code Words in Euphemistic Hate Speech finds code words with static Google News vectors and a seed word, and loses slang such as "leppos" entirely to out-of-vocabulary gaps. This motivates training the encoder on in-community text rather than relying on general vectors. Unmasking the Hidden Meaning shows that implicit hate sits tangled with neutral text in standard BERT space and fixes this with supervised contrastive training on target labels. We do not use labels, so we accept that some coded senses may not separate. The synthetic-injection test below measures how often that happens. The domain-splitting results in Optimal Splitting of Language Models are informative only as a caution: specialization helps in-domain but can erode general behavior. We therefore compare adapted and unadapted encoders rather than assume adaptation helps.

Research Program

The data come from two sources. The pre-ban side is Reddit r/The_Donald for 120 days before the ban, matching the window in the migration study. The post-ban side is c/TheDonald in iDRAMA-Scored-2024, which holds 56.8M Scored posts, 41.7M of them from c/TheDonald, and identifies 58 migrant communities. Where Reddit-side history exists, other Scored migrant communities serve as replications. The control is a set of politically adjacent subreddits that were not banned, observed over the same calendar windows. The control absorbs macro events such as the 2020 election, which the migration paper flags as a confound. Users are matched by exact username, which defines "migrants," "stayers-who-left," and "newcomers" (Scored users with no Reddit match).

The encoder is BERT-base with continued pretraining on pre-ban community text only. Post-ban text is excluded so the model cannot learn the post-ban senses in advance. Target words are the union of the fixation dictionaries, the frequent dogwhistle surface forms from the glossary, and the top words by frequency change. For each target word $w$, all contextual usages from both periods and the control are pooled and clustered into senses. The number of clusters is chosen per word by silhouette score. We use joint clustering because the cross-period Hungarian matching variant in SST-BERT complicates per-user attribution.

The decomposition works as follows. Let $P_{G,t}(s \mid w)$ be the sense distribution of word $w$ among user group $G$ in period $t$. The groups are all pre-ban posters ($A$), matched migrants ($M$), and all post-ban posters ($B$). The observed shift $D(P_{A,\text{pre}}, P_{B,\text{post}})$, with $D$ the Jensen-Shannon distance, is traced along the path

$$P_{A,\text{pre}} \rightarrow P_{M,\text{pre}} \rightarrow P_{M,\text{post}} \rightarrow P_{B,\text{post}}.$$

The first leg is selection (how migrants already differed before the ban). The second is within-migrant change. The third is newcomer composition. JSD is not additive, so we report each leg's distance and also an additive version using total variation. We check sensitivity to the order of the path. Within-migrant change is then compared against the same leg computed for a matched cohort in the control subreddits, in difference-in-differences style. Significance comes from a user-level permutation null: period labels are shuffled within users, following the rectified-change logic of the social-network shift paper. Per-user sense distributions are weighted by usage count, and super-users are capped so that a handful of accounts cannot dominate.

Baselines

The naive reference point is the community-level SST-BERT comparison: cluster usages of each word, then compute JSD between all pre-ban Reddit posts and all post-ban Scored posts, with no user matching. This is the "obvious" study the file's Gap 1 describes. Every claim we make must be a claim this baseline gets wrong. On its native benchmark, SST-BERT reports 0.637 overall accuracy on SemEval-2020 Task 1 Subtask 1 (Hungarian-matching variant), and 0.185 on Subtask 2 for the same variant. We will reproduce these numbers with our clustering code before any migration analysis. Our method beats this baseline if it recovers the injected selection/within-user split in the synthetic test where community-level JSD cannot, and if its ranking of within-user shifts agrees better with human sense annotations.

The strongest cheaper competitor is static embeddings with the rectified-change null from Semantic shift in social networks. These are trained separately on migrant-only pre- and post-ban text and aligned with Procrustes rotation. This gives static methods the same composition control by restricting to matched users. It is the real threat, because user restriction rather than contextualization may be doing most of the work. It runs on identical user subsets, target words, and permutation budget.

A third baseline is centroid drift of the released INSTRUCTOR-large post embeddings in iDRAMA-Scored-2024. We compute the mean embedding of posts containing $w$, per group and period, and apply the same decomposition. This tests whether word-level sense clustering adds anything over sentence vectors that already exist for nearly 48M posts.

The key ablation replaces the community-adapted encoder with off-the-shelf BERT-base, keeping everything else fixed. A second ablation drops the control community.

There is no reported result on Lacuna for semantic change across a deplatforming event. All migration-side comparisons are therefore new. The only external anchor is the SemEval number above.

Why not just…?

Why not just run SST-BERT on pre- versus post-ban text? Because the post-ban population is a filtered subset of the pre-ban one. The migration study reports roughly 55% fewer posts and 65% fewer active users, with the remaining users far more active. A community-level shift therefore conflates who stayed with what changed. The synthetic test shows this directly. Shifts planted only as a reweighting of existing users' senses will register as "change" under the baseline, and our first leg should absorb them.

Why not just use static embeddings on matched users? This may work nearly as well for words with one dominant sense, and we take that possibility seriously. Static vectors give one point per word per period. They cannot represent a word that keeps its old sense and gains a coded one, which is the dogwhistle pattern. They also lose rare slang to out-of-vocabulary gaps, as the euphemism paper found. The head-to-head is split by word type: polysemous glossary dogwhistles versus monosemous fixation terms. If static matched-user shift ties us on the dogwhistles, we will report that contextualization is unnecessary and that the contribution is the decomposition itself.

Why not just track Perspective scores or dictionaries, as the migration literature does? These answer a different question, namely how toxic the posts are. They are documented to miss coded speech: the dogwhistle paper shows Perspective scores collapse under substitution, and the Parler paper shows general detectors fail on fringe text. We still compute them so readers can see where the semantic decomposition disagrees with toxicity trends.

Why not just prompt a large language model? In the dogwhistle paper, GPT-3 recalled only 12.9% of informal internet-born dogwhistles. Running a model like that over tens of millions of posts is also impractical. We use an LLM only as a secondary annotator on sampled usages, checked against humans.

Proposed Experiments

The decisive check is synthetic injection with known ground truth. We take real Reddit and Scored text and choose control words. Into one group of words we plant an artificial new sense only in post-ban posts of matched migrants, by swapping in contexts from an unrelated word. This is within-user change. Into a second group we plant it in the pre-ban posts of migrants only, which is pure selection. A third group gets it only from newcomers. Each method must attribute each injected shift to the correct leg. This experiment discriminates between our method and the community-level baseline even if no real coded language exists.

On real data, we report three things. The first is the share of total shift explained by each leg across fixation and glossary words. The second is the list of words with significant within-migrant, control-adjusted change. The third is whether that list is enriched for glossary dogwhistles relative to the frequency-matched words from the target-selection step. For validation, two annotators label the sense of 200 sampled usages per top-ranked word. Agreement with cluster assignments and with the estimated per-user change is the external metric. Replication on additional Scored migrant communities tests whether the selection share is stable across communities or community-specific, echoing the r/The_Donald versus r/Incels asymmetry.

The prediction would be falsified in either of two ways. The within-migrant leg could be indistinguishable from the control cohort for all words, in which case post-ban "drift" is selection plus background. Or the adapted encoder could fail the injection test at rates no better than static matched-user embeddings. Either outcome would be a clear, publishable correction to how migration effects are currently measured.