Claims that state propaganda "says different things on different platforms" now rest almost entirely on automatic classifiers. Park and colleagues report that Russian state outlets push Security & Defense frames to foreign audiences on Twitter and Legality and stability frames to domestic audiences on VKontakte (Challenges and Opportunities in Information Manipulation Detection). The same paper shows that the classifier behind such findings falls from 67.5 to 33.5 F1 when moved from its training benchmark to wartime Russian social media. That creates a measurement problem. If the classifier makes different mistakes on short English tweets than on long Russian VKontakte posts, then part of any measured "platform difference" in framing is really a difference in classifier error. To our knowledge, no work in this area checks for it.
We propose to build framing classifiers whose errors are measured and corrected separately for each platform. The method is knowledge distillation, meaning a small "student" model trained to imitate the soft, probabilistic outputs of larger "teacher" models. Here the teachers are a committee of large language models and existing framing classifiers. We then use the corrected instrument to re-test the published claims about platform bifurcation and the "chilling effect" on user comments. Distillation is not the goal in itself. It is what makes post-level labels for tens of millions of posts affordable, and those labels are what engagement regressions need. The platform-stratified error audit is what makes the resulting comparisons trustworthy.
Core Claim
A distilled student trained on soft labels from a diverse teacher committee, and audited with a small human-coded sample per platform, can produce platform-level frame prevalence estimates with known, correctable bias at a small fraction of the cost of running large models on full corpora. We expect, and will test, that at least some published platform contrasts shrink or change sign once differential error is corrected. We also expect the student to support post-level analyses that sampling-only approaches cannot power.
The closest prior work uses distillation for propaganda in a different way and for a different aim. Lei and Huang distill two discourse-structure teachers into a Longformer student to find propaganda sentences in English news, reaching 65.21 sentence-level F1 (Discourse Structures Guided Fine-grained Propaganda Identification). That work is single-domain, single-language, and platform-agnostic, and it treats accuracy rather than comparative validity as the target. FedACK distills across platforms and languages, but for account-level bot detection, not content framing (FedACK). PCoT shows that two-stage persuasion reasoning improves large-model disinformation detection, but it never compresses that reasoning into a cheap model (PCoT). Our contribution is the error audit across platforms and the substantive re-test that the audit enables.
Research Program
Teacher committee. The committee mixes model types that are known to disagree. It includes the XLM-RoBERTa classifier trained on the Media Frames Corpus, the 15-frame taxonomy used by Park et al.; the lexicon-embedding framing method of Sinelnik and Hovy; and two or three instruction-following language models prompted with the frame codebook, including one open model such as Llama 3.1 8B so the pipeline stays reproducible.
The disagreement is deliberate. Sinelnik and Hovy find that lexicon and transformer framing methods agree with Krippendorff's alpha of only 10.2 to 13.7 (Narratives at Conflict). Multi-teacher distillation work argues that combining teachers cancels uncorrelated errors (Amalgamating Diverse Perspectives from Multiple Teacher Models; Learning from All). We use the committee's averaged soft distribution as the target. We weight each teacher per platform by its accuracy on a small development audit, so that a teacher that is good on Twitter but poor on Odnoklassniki does not dominate there.
Student and loss. The main student is a base-size multilingual encoder. Frames overlap: Security & Defense and Legality co-occur in wartime posts. For this reason we compare standard distillation with decoupled knowledge distillation. Zhao et al. show that the standard loss suppresses information about non-top classes by a factor of $(1-p_t)$ when the teacher is confident, and that separating the two terms fixes this (Decoupled Knowledge Distillation). The overlap structure between frames is exactly that kind of non-top-class information.
A second student is an Aug-GAM, a linear model over n-grams whose phrase weights are scored by a pretrained language model and stored as a lookup table (Emb-GAM). We fit it with platform-interaction terms so analysts can read off which phrases carry a frame on one platform but not another. This responds directly to Vestel and Degaetano-Ortlieb's argument that policy users need transparent linguistic indicators (Interpretable Models for Detecting Linguistic Variation in Russian Media). Emb-GAM itself notes that n-gram students lose long-range context, so we expect this student to trail the encoder. The gap between the two will be reported.
Platform audit and correction. Native-speaker coders, following the Media Frames Corpus codebook, label a stratified audit sample from each platform: Twitter and VKontakte from VoynaSlov, and VKontakte, Odnoklassniki and Telegram from WarMM-2022. Before sampling, we remove duplicates, because WarMM shows that duplicate astroturfing posts inflate Odnoklassniki counts (WarMM-2022). From each platform's audit we estimate a confusion matrix $M_p$. Its entry $(i,j)$ is the probability that the student predicts frame $i$ when the true frame is $j$ on platform $p$. Writing $q_p$ for the student's predicted frame shares on platform $p$, the corrected true shares $\pi_p$ satisfy
$$q_p = M_p \, \pi_p .$$
We solve for $\pi_p$ with a non-negativity constraint and bootstrap over the audit sample to get confidence intervals. The quantity of interest is the corrected difference $\pi_{\text{Twitter}} - \pi_{\text{VK}}$ for state outlets, compared against the uncorrected difference. For post-level regressions, such as the mixed-effects models of comment counts against frames behind the chilling-effect finding, we propagate label uncertainty by multiple imputation from the student's calibrated probabilities.
Baselines
The naive reference point is the instrument behind the published findings: XLM-RoBERTa fine-tuned on the Media Frames Corpus and applied unchanged. Park et al. report 33.5 F1 on VoynaSlov against 67.5 on the source benchmark. We rerun it on our platform audits and report F1 separately per platform, since the size of the gap between platforms is the point. Beating it means higher per-platform F1 and, more importantly, a smaller spread in error between platforms.
The strongest competitor is direct prompting of the best teacher in the committee. We add a PCoT-style two-stage variant that first names the rhetorical strategy and then assigns the frame. PCoT reports 0.846 F1 for GPT-4o Mini on binary disinformation detection, but that is a different task. We found no reported result on Lacuna for large-model framing classification on VoynaSlov or WarMM. We therefore run it ourselves on the same audit samples, with the same codebook and the same prompt-tuning budget we give to the student's hyperparameters. The student "beats" it if it comes within a few F1 points per platform at a large reduction in cost per post. The FENICE result gives a reasonable expectation: a distilled T5 reached 70.0% balanced accuracy against 72.7% for its GPT teacher (FENICE). The student also beats it if it yields tighter intervals on cell-level estimates at equal spend.
Three ablations isolate the proposal's ingredients. The first is single-teacher distillation from the best teacher, which tests whether the committee matters. The second is fine-tuning on the committee's majority-vote hard labels, which tests whether soft labels matter. The third is the full student without the per-platform correction, which tests whether the audit changes substantive conclusions. All students share architecture, training data, and a fixed hyperparameter search budget. Because DKD is sensitive to its two loss weights, those weights are tuned within the same budget rather than separately.
For propaganda techniques we add a secondary comparison on ZenPropaganda, where RoBERTa-CRF reaches 0.173 F1 over 36 techniques and 0.42 over six meta-classes (ZenPropaganda). We test whether decoupled distillation from a large-model teacher improves the rare classes.
Why not just…?
Why not just prompt a large model on a random sample and skip distillation? This is the most serious alternative, and for some questions it may be enough. If the only question is the overall frame share per platform, a few thousand large-model labels with a human calibration set may give intervals as tight as ours. We will test this head-on by giving both approaches the same budget. Sampling weakens when the analysis splits into many cells: 43 outlets, several platforms, weekly bins, and engagement models at the post level. In that regime per-cell samples become tiny, while a cheap student can label every post across VoynaSlov's 38 million. If sampling matches us even at the cell level, we will report that the audit-and-correct procedure, not distillation, carries the contribution.
Why not just fine-tune XLM-RoBERTa on more in-domain labels? Collecting enough human labels across five platform settings to fix a 34-point drop is expensive. Fine-tuning also still yields a single model whose errors can differ by platform. We include an equal-annotation-budget fine-tuning baseline, in which the human labels spent on audits go to training instead. The comparison shows whether those labels are better spent on training data or on measuring error.
Why not just use lexicons or topic models, as media studies often do? Park et al. report that structural and contextual topic models give conflicting results on this data. WarMM shows that keyword counts are inflated by duplicate posts. DeFaktS finds TF-IDF trailing BERT-German, at 0.76 against 0.86 binary F1 (DeFaktS). We keep a lexicon method in the teacher committee and score it against the audits, so its per-platform error is measured rather than assumed.
Proposed Experiments
The first decisive check is whether differential error exists at all. We will compare per-platform F1 and confusion matrices for the naive classifier and for direct large-model prompting. If errors turn out to be equal across platforms, the correction is unnecessary, and the paper becomes a cost and validity study of distilled framing classifiers.
The second check re-estimates the Twitter-versus-VKontakte frame contrast for state outlets, and the Odnoklassniki-versus-Telegram pro-war contrast from WarMM, with and without correction. For each contrast we report whether its sign and its confidence interval survive.
The third check reruns the chilling-effect regression on corrected post-level labels with multiple imputation, and asks whether the drop in comments on posts with Political and Fairness frames persists.
The fourth check compares the cost and interval width of the student against prompting on a sample, at matched budgets, across cell granularities from platform-level shares down to outlet-by-week cells. The aim is to locate where distillation starts to pay off.
Finally, we apply the student without retraining to German Twitter in DeFaktS. This tests whether the committee-distilled student degrades less than single-teacher students outside the Russian setting.
References
- Challenges and Opportunities in Information Manipulation Detection: An Examination of Wartime Russian Media
- Discourse Structures Guided Fine-grained Propaganda Identification
- FedACK: Federated Adversarial Contrastive Knowledge Distillation for Cross-Lingual and Cross-Model Social Bot Detection
- PCoT: Persuasion-Augmented Chain of Thought for Detecting Fake News and Social Media Disinformation
- Narratives at Conflict: Computational Analysis of News Framing in Multilingual Disinformation Campaigns
- MACPO: Weak-to-Strong Alignment via Multi-Agent Contrastive Preference Optimization
- Learning from All: Concept Alignment for Autonomous Distillation from Multiple Drifting MLLMs
- Decoupled Knowledge Distillation
- Emb-GAM: an Interpretable and Efficient Predictor using Pre-trained Language Models
- Interpretable Models for Detecting Linguistic Variation in Russian Media: Towards Unveiling Propagandistic Strategies during the Russo-Ukrainian War
- Wartime Media Monitor (WarMM-2022): A Study of Information Manipulation on Russian Social Media during the Russia-Ukraine War
- FENICE: Factuality Evaluation of summarization based on Natural language Inference and Claim Extraction
- ZenPropaganda: A Comprehensive Study on Identifying Propaganda Techniques in Russian Coronavirus-Related Media
- DeFaktS: A German Dataset for Fine-Grained Disinformation Detection through Social Media Framing