Borrowing Strength Across Communities: Graph-Coupled Estimates of Word Meaning Divergence in Small Online Communities

Words like "moral", "refugees", or "freedom" mean measurably different things in different communities at the same moment in time. This is called synchronic semantic divergence. Today's standard way to measure it trains a separate word-embedding space for each community and then rotates the spaces into alignment. That works for giant forums with millions of comments. It breaks for exactly the communities that are most interesting to study: small, fringe, or newly formed groups, whose private meanings are hardest to see. We propose to estimate each community's word meanings as a shared base vector plus a community-specific deviation. The deviations are tied together by the community interaction graph, for example communities that share many users. A small community then borrows statistical strength from its structural neighbors instead of from the whole internet. It keeps its own meaning wherever its own text gives enough evidence to override them.

The claim is concrete and falsifiable. At a fixed, small amount of text per community, graph-coupled deviations should recover the divergences that full-data estimates reveal more accurately than three alternatives: separate-and-align, fine-tuning a global space, and uniform shrinkage toward the global mean. The advantage should vanish when the graph is shuffled. The same machinery treats domain specialization as a special case of divergence. A technical subcommunity is simply a community whose deviations are large for jargon and near zero for everyday words.

Core Claim and Mechanism

Let $u_w$ be a shared vector for word $w$, trained on the pooled corpus. Let $d_{c,w}$ be the deviation of community $c$, so that community $c$'s meaning of $w$ is $u_w + d_{c,w}$. Let $A_{cc'}$ be a nonnegative weight on the community graph, measuring for instance the fraction of shared commenters between communities $c$ and $c'$. We fit

$$\min_{u,d}\; \sum_c \mathcal{L}_{\text{SGNS}}(u + d_c;\, D_c) \;+\; \lambda \sum_{c,c'} A_{cc'} \sum_w \lVert d_{c,w} - d_{c',w}\rVert^2 \;+\; \mu \sum_{c,w} \lVert d_{c,w}\rVert^2 .$$

The terms are as follows. $\mathcal{L}_{\text{SGNS}}$ is the skip-gram negative-sampling loss (the word2vec objective) on community $c$'s corpus $D_c$. The $\lambda$ term pulls neighboring communities' deviations together. The $\mu$ term shrinks all deviations toward the shared meaning. The divergence of word $w$ in community $c$ is read off $d_{c,w}$ directly. Because every community lives in one parameterization, no post-hoc alignment step is needed.

The multidimensional variant replaces $A_{cc'}$ with a coupling $A^{(k)}_{cc'}$ that is learned separately for each topic cluster $k$ of the vocabulary. This lets two communities stay tightly coupled on sports words and uncoupled on economic words. The learned couplings are themselves an output: a map of where connected communities agree and where they split.

We deliberately smooth deviations rather than full vectors, and we do it inside the training objective rather than by repeated message passing. GeoGNN shows that repeated Euclidean neighbor-averaging of language-model embeddings drags them off their curved manifold and blurs class distinctions within three to four layers. That result is a constraint on our design. It is also a warning that naive graph smoothing could erase exactly the divergence we want to measure. Our $\mu$ and $\lambda$ are tuned so that a community's own text can always outvote its neighbors. The key ablations test whether that balance holds.

Relation to Prior Work

Three lines of work motivate the proposal. Aligning Multidimensional Worldviews trains separate skip-gram spaces per subreddit, aligns them with multi-way canonical correlation on anchor words, and finds functional equivalents such as "Trump" and "Clinton" across partisan forums. Words are Malleable aligns per-party spaces from UK Hansard and compares nearest-neighbor sets. It shows that the diachronic laws of conformity (frequent words shift less) and innovation (polysemous words shift more) also hold across viewpoints. SemAxis fine-tunes word2vec per community and projects words onto antonym-defined axes, such as respect versus disrespect.

All three treat communities as disconnected bags of text, and all three need large per-community corpora. We keep their measurement tools (alignment accuracy, neighbor overlap, axis projections) as readouts. We change the estimator so that community structure supplies a prior.

A second group supplies the idea that structure should shape text representations, but at a different granularity. GLEM co-trains a language model and a graph neural network by exchanging pseudo-labels. GraphFormers nests graph attention inside Transformer layers so neighbors disambiguate a node's text. UniGraph pretrains a text-plus-graph foundation model that assumes language is a shared bridge across domains. Understanding Politics via Contextualized Discourse Processing builds a politician–document–issue graph and predicts stances at the politician level.

These models all apply one text encoder across the whole graph and target node or document labels. None produces word-level, community-specific meanings, and none asks whether the graph helps estimate them. We use structure one level up: the graph governs how word meanings are allowed to differ, not how node labels are predicted.

The domain-specialization literature frames the third role. Cancer Hallmark Text Classification and Brundlefly at SemEval-2016 show that in-domain embeddings beat general ones. They also note the cost of training separate domain spaces and the loss of general fluency. The additive parameterization $u_w + d_{c,w}$ addresses that siloing directly. It pays only for the words that actually specialize and leaves the rest shared. We test this on gaming communities, whose jargon-heavy vocabularies make them a small-scale domain setting.

Research Program

The main corpus is the Reddit comment collection used by Aligning Multidimensional Worldviews: over one billion comments across 32 subreddits. It spans political, gender, and gaming communities, which gives a multi-polar setting rather than a binary one.

The community graph is built from shared-commenter overlap. If commenter identities are not recoverable from the release, we fall back to cross-community reply links. We will not build the graph from lexical similarity, because that would leak the quantity being measured into the prior.

The central experimental device is controlled downsampling. Large communities give full-data reference divergences. We then subsample each community to budgets of roughly ten thousand, one hundred thousand, and one million tokens and ask how well each estimator recovers the reference.

To avoid rewarding any one estimator's biases, the reference is computed two ways. The first is Milbauer-style separate training with MultiCCA alignment on full data. The second is a method-agnostic nearest-neighbor overlap score in the style of Azarbonyad et al. We report agreement with both.

Beyond recovery, we analyze three things. First, whether the learned per-topic couplings reproduce known structure, such as partisan forums coupled on gaming vocabulary and uncoupled on political vocabulary. Second, whether the conformity and innovation laws still hold in the estimates for small communities, as a sanity check on plausibility. Third, how the deviation norm splits between jargon and everyday words in the gaming communities, which tests the domain-specialization reading.

Baselines

The naive reference point is a single pooled skip-gram space with no community deviations. Every word gets divergence zero, and random or frequency-based rankings are used for divergence-ranking tasks. Any method that cannot beat this at small budgets is measuring noise.

The strongest existing method for this exact question is separate training plus MultiCCA alignment from Aligning Multidimensional Worldviews. On full data it reports Acc@10 of 0.8924 for alignment with 5,000 anchor words and MAP of 0.3318 for polarized political word discovery. We rerun it on the same subsamples, with the same vocabulary and embedding dimension and the same anchor-selection procedure. We expect its full-data numbers to be near those values and its small-budget numbers to degrade. Beating it means higher polarized-word MAP and higher reference-recovery correlation at the small budgets, with no loss at full data.

The fine-tuning baseline follows SemAxis: start from the pooled space and fine-tune on each community's text. Because initialization acts as an implicit prior toward the global meaning, this is the most dangerous competitor. We give it the same hyperparameter search budget as our method, including the number of fine-tuning epochs, which controls its effective shrinkage. It is run on identical subsamples.

The contextual baseline averages BERT layer-10 representations of each target word within each community. This follows One Word, Two Sides, which found that layer most sensitive to stance and reported global pairwise stance-discrimination accuracy of 0.99. This baseline needs no per-community training at all, so it tests whether pretrained knowledge already solves the small-data problem.

Two ablations isolate the key ingredient. Uniform shrinkage sets $\lambda = 0$ and keeps only $\mu$. Shuffled-graph coupling permutes community identities in $A$ while preserving degree, so the graph keeps its statistics but loses its meaning. If our method beats the shuffled-graph ablation at matched hyperparameters, the gain comes from the graph structure rather than from regularization in general.

Why not just…?

Why not just fine-tune a general embedding on each small community? This is the real threat. Initialization from the pooled space already shrinks toward shared meaning, and it may be enough. The difference we predict is specific. Fine-tuning shrinks every community toward the same global center. Graph coupling shrinks a small far-right forum toward other right-leaning forums rather than toward all of Reddit, so it should preserve divergences that the global center erases. If the fine-tuning baseline matches us at every budget once its epoch count is tuned, the graph adds nothing, and we will report that.

Why not just average contextual BERT vectors per community? Contextual models carry stance traces even in trimmed sentences, per One Word, Two Sides. However, that study measures whether stances are separable, not whether a community's functional substitutions (who plays "Clinton" for whom) are recoverable. BERT's pretraining meaning may also dominate rare community-specific senses. The downsampling experiment settles this directly: contextual averaging has no per-community training cost, so if it matches at small budgets, it wins on simplicity.

Why not just append a community tag to the input? A single tag gives every word the same community offset. It cannot express that two communities agree on sports and disagree on economics, which Aligning Multidimensional Worldviews documents. It also provides no sharing between related small communities. Our uniform-shrinkage ablation is essentially a per-community free offset with a prior, so it bounds how well a tag-style model could do.

Proposed Experiments and Decisive Checks

The primary experiment is the downsampling recovery curve on Reddit. Each estimator's divergence scores at small budgets are correlated (Spearman) with both full-data references. We also report polarized-word MAP against the Milbauer gold list and functional-equivalent Acc@10.

The decisive check is a specific ordering at the ten-thousand and one-hundred-thousand token budgets. Graph coupling should beat uniform shrinkage and the shuffled graph. It should match or beat SemAxis-style fine-tuning. At full data, the curves should converge.

The secondary check is on UK Hansard, at the party level with speaker-level subsampling. We test whether divergence-based document expansion with our estimates keeps the ideology-classification gain that Words are Malleable reports: F1 rising from 0.76 to 0.81. We use its 24 contested words as a human-interpretable probe.

The domain check uses the gaming communities. The deviation norm should concentrate on game-specific jargon. Cross-game functional equivalents, such as League of Legends to Dota 2 characters, should be recoverable at budgets where separate alignment fails.

A negative result on the shuffled-graph ablation would mean that interaction structure does not predict shared meaning. That would be a useful finding about the sociolinguistics itself, and we will report it as such.