Who Is the Joke About? Intervenable Relational Concepts for Hateful Meme Moderation

A hateful meme usually says nothing hateful on its own. The caption "love the way you smell" is harmless and a photo of a skunk is harmless. The meme becomes hateful when the caption is read as being about a person from a protected group, who is either shown in the image or implied by it. Current detectors are opaque neural networks that fuse image and text features and output a score. A moderator cannot see which reading the model chose, and cannot fix it when the reading is wrong. We propose a detector whose only route to a decision passes through a small set of objective, checkable facts. The facts cover who is depicted, what kind of claim the text makes, and which depicted entity the text is about. A learned rule layer then decides harm from these facts. Because each fact is something a moderator can verify by looking, a moderator can correct a wrong fact and watch the decision update.

The design rests on a negative result. A large user study of concept bottleneck models (networks forced to predict human-named concepts before the label) found that they help human decision-makers only when users can correct the concepts, and only when the concepts are objective. With fuzzy concepts such as "urgency" in phishing emails, they gave no advantage over a bare label (Are Concept Bottleneck Models Effective as Decision-Support Systems?). Concepts like "offensive tone" or "sarcastic framing" are just as fuzzy. Our claim is that hate in memes can be broken down so that the subjective judgment lives only in the learned rule layer. Everything a human is asked to check is then factual, and the cross-modal link between text and image is itself one of those facts.

Core Claim

We claim that a bottleneck of objective relational atoms recovers most of the accuracy of a black-box detector on Hateful Memes and MAMI. Relational atoms here are yes/no facts about single entities and about pairs of entities across modalities. The central pair-level atom is "the text refers to this image entity." We further claim that this bottleneck behaves differently from a black box in two measurable ways. First, on the benign-confounder pairs of Hateful Memes, where a minimal edit to one modality flips the label, the flip should be carried by a change in a small number of atoms. Restoring those atoms by hand should restore the original prediction. Second, in a user study, correcting these atoms should improve moderator accuracy over label-only support. Correcting fuzzy "tone" concepts on the same memes should not. If both hold, the decision-support result from emails and birds carries over to content moderation in a specific form: the relational facts are what make intervention work.

Relation to Prior Work

Three lines of work motivate this proposal by showing that explanation and structure exist in meme moderation, but never as an intervenable decision path. What is Beneath Misogyny projects CLIP-ViT features into Llama-3-8B and then prompts a second model to write rationales. BANMIME fine-tunes vision-language models with chain-of-thought (step-by-step text reasoning) from metaphor clues to harm labels. Both produce fluent text. Neither guarantees that the text caused the decision, and neither lets anyone edit an intermediate step. Multimodal and Explainable Internet Meme Classification explains by retrieving similar labeled memes. It reports that prototype learning collapsed into nearly one prototype per training meme because memes are so diverse. That finding constrains us: harm structure has to be compositional, not memorized.

Two papers come closest to supplying structure. Unintended Bias Detection and Mitigation in Misogynous Memes builds entity–object scene graphs and uses contrastive learning to stop models from keying on words like "kitchen." The graph then feeds an opaque fusion head, so its structure is discarded at decision time. We keep it as the decision path. Are you a hero or a villain? frames entities as hero, villain or victim, which is close in spirit to our atoms. It implements the framing with sentiment lexicons and word-distance heuristics and reaches macro F1 23.855. We learn the atoms from multimodal features instead.

The method comes from the interpretability side. Concept Bottleneck Models supply the basic input-to-concepts-to-label design and test-time intervention. Multimodal Concept Bottleneck Models map image and text into a shared concept space on CLIP, but each concept is scored on one modality independently. That works for object and scene recognition, where the caption describes the image. It cannot express "the text is about this depicted person," and that is exactly what separates a hateful meme from its benign confounder. Relational Concept Bottleneck Models supply the missing piece: concepts as ground atoms over entities and relations, with a logic-based reasoner and atom-level intervention. So far they have been tested only on Tower of Hanoi, citation graphs and knowledge bases. PCBM-ReD shows that sparse decompositions of frozen encoder features can match end-to-end accuracy within 0.41% on image benchmarks. We use its sparse-autoencoder route as a check that our hand-specified atoms are not missing large amounts of signal. Finally, Uncertainty-Guided Modal Rebalance shows that text tends to dominate training and suppress the image modality. This constrains our design: a bottleneck could quietly use only text atoms, and we must measure whether it does.

Research Program

Each meme is parsed into entities. Image entities come from a detector run on the meme. Text mentions come from the OCR caption, including pronouns and implied addressees such as "you." The bottleneck consists of three groups of atoms. Unary image atoms describe a detected entity, for example "depicts a person," "visible religious attire," "animal," "domestic setting." Unary text atoms describe a mention or the caption as a whole, for example "makes a claim about smell or dirtiness," "issues a command," "compares to an object or animal," "references violence." Binary cross-modal atoms record whether a text mention refers to an image entity, and whether the caption contradicts what the image shows. Every atom is chosen to pass a simple test: two annotators who look at the meme should agree on its value without having to decide whether the meme is hateful. We will measure agreement on a pilot set and drop atoms that fall below the label-level agreement reported for Hateful Memes (Cohen's κ of 68.4).

Writing $a(x) \in \{0,1\}^m$ for the vector of atom values on meme $x$, the classifier is

$$\hat{y} = g\big(a(x)\big),$$

where $g$ is a sparse rule layer in the style of the Deep Concept Reasoner used by Relational CBMs. Every input to $g$ is a hard yes/no atom. This blocks the leakage problem, in which soft concept scores smuggle unnamed features past the bottleneck to the classifier. Atom predictors are small heads on a frozen BLIP-2 or CLIP encoder. They are trained on three sources: a few thousand human-labeled memes; weak labels from a vision-language model answering one atom question at a time; and existing annotations. The Constraint 2022 hero/villain/victim roles supervise framing atoms, and the BANMIME metaphor bounding boxes supervise binding atoms. The rule layer is trained on the harm labels with the atom predictors held fixed.

Two analyses go with the model. The first is a leakage audit. We probe whether the frozen encoder's residual (the part not captured by the atoms) still predicts the label, and we compare our atoms against a PCBM-ReD-style sparse dictionary. This shows how much signal the named atoms miss. The second is a modality audit, which reports what fraction of the rule layer's decisions depend on at least one image or cross-modal atom. This tests whether the text dominance documented by UMR reappears inside the bottleneck.

Baselines

The naive reference point is the text-only BERT from The Hateful Memes Challenge, which reports 62.8% accuracy and 69.00 AUROC. The same paper reports Visual BERT COCO at 69.47% accuracy and 75.44 AUROC, and human accuracy at 84.70%. We rerun both baselines rather than relying only on the quoted figures.

The strongest black-box competitor on Hateful Memes is UMR on the same frozen encoder we use. It is the best-documented fix for the modality imbalance our bottleneck must also survive. The file reports only relative gains for UMR (+10.68% AUROC with CLIP, +12.22% with BLIP-2); no reported absolute result was found on Lacuna for UMR. We will therefore retrain it ourselves on the same backbone, data splits and hyperparameter search budget. On MAMI the reference numbers are macro F1 73.84% for the MisoM baseline reported with CTXSGMNet, and 0.701 for the example-based CLIP+BERTweet model of Thakur et al. CTXSGMNet's bias metrics, FPRED falling from 12.35% to 6.98%, set the fairness bar. On WBMS, the Llama-3-8B+CLIP model reports F1 0.89.

The explanation-producing competitor is a Gemma-3-12B chain-of-thought model fine-tuned as in BANMIME, trained on the same memes. Three ablations isolate the key ingredient. The first is an independent-concept bottleneck in the MM-CBM style: the same unary atoms, no cross-modal binding atoms, same rule layer. The second is our model with soft rather than hard atoms, which tests leakage. The third is a fuzzy-concept bottleneck whose concepts are "offensive tone," "sarcasm," "dehumanizing framing" and similar, with the same backbone and annotation budget. "Beating" UMR does not mean higher AUROC. It means AUROC within three points of UMR together with a significant intervention advantage that UMR cannot offer. Against the independent-concept ablation we expect a clear AUROC gain concentrated on the confounder subsets.

Why not just…?

Why not just fine-tune a vision-language model to explain its answer in text? It may well be more accurate, and we treat that as a real threat. BANMIME and WBMS show fluent rationales, but they also show confusion among shaming, objectification and stereotype, and misreadings of affectionate or ironic framing. More importantly, a free-text rationale cannot be corrected and re-run. The decision-support study found that non-interactive explanations gave no reliable gain over a bare label. The user study below includes this model's rationales as a condition. If moderators do as well with those rationales as with editable atoms, our case fails.

Why not just have a vision-language model answer concept questions and fit a linear head? This is essentially our atom extractor, so the honest question is whether the relational atoms and the rule layer add anything. The MM-CBM-style ablation answers it. If independent unary atoms with a linear head match our model on the confounder pairs, the relational structure is unnecessary and we would report that.

Why not just retrieve similar labeled memes as precedent? Thakur et al. found that prototypes degenerate on memes. Retrieval by embedding similarity also cannot tell a hateful meme from its confounder, because the two are nearly identical by construction. We include the k-nearest-neighbor model and report its accuracy on confounder pairs specifically.

Why not just debias a black box and add saliency maps? CTXSGMNet already reduces identity shortcuts, but the authors note that it overcorrects on subtle cases and gives no way to inspect why. We compare fairness gaps directly. If our model does not at least match CTXSGMNet's FPRED, interpretability has been bought at the cost of fairness.

Proposed Experiments and Decisive Checks

We evaluate on Hateful Memes (AUROC and accuracy, broken down by multimodal hate, unimodal hate, and text versus image confounders), MAMI subtasks A and B (macro F1, FPRED, FNRED), and WBMS and BANMIME (F1) for transfer to new domains and to low-resource settings.

The first decisive check is faithfulness on confounder pairs. For each pair in the Hateful Memes development set, we find the atoms whose values differ between the two memes. We then set those atoms in the hateful meme to the values they take in the benign meme and record whether the prediction flips. We report the flip rate and the number of differing atoms. A faithful relational bottleneck should flip most predictions with one or two binding or predicate atoms. If the flip instead requires many unary atoms, the model is matching surface features.

The second check is a preregistered moderator study following the protocol of the decision-support paper. It has four conditions: label only, chain-of-thought rationale, editable objective atoms, and editable fuzzy concepts. We oversample memes on which the model is wrong. The prediction is that only the editable objective atoms raise accuracy on error cases. If fuzzy concepts help equally, our explanation for when intervention works is wrong.

The third check is the modality audit. If the rule layer rarely uses image or binding atoms, the bottleneck has reproduced text dominance, and we will test whether UMR-style rebalancing applied to the atom heads repairs this.