FoodReact-AR: Learning Audience Reactions that Zero-Shot LLMs Miss in Arabic Food-Insecurity Discourse

Wajdi Zaghouani, Mabrouka Bessghaier, Salma Azouzi, Shimaa Amer IbrahimarXiv 2026 · arXiv

TL;DR

Researchers have introduced FoodReact-AR, a new dataset and evaluation protocol that highlights a significant blind spot in Large Language Models (LLMs): the ability to detect derisive audience reactions. While zero-shot Arabic LLMs are excellent at identifying "Grief" in food-insecurity discourse, they almost entirely fail to recognize "Dismissal" (sarcasm or mockery), a signal that simple supervised models can easily learn from text and metadata.

What Problem Does This Paper Address?

In social media analysis, we often assume that the sentiment of a post tells us how people feel. However, "what is said" is not always "how it is received." A government ministry might post a neutral, factual update about a food budget, but the audience might respond with a flood of "Haha" (laughing) reactions—a clear sign of derision or institutional distrust.

Current NLP pipelines, especially those relying on zero-shot LLMs for annotation, tend to read text at face value. This paper investigates whether these models miss the "hidden" signal of audience reception in Arabic food-insecurity discourse, specifically across the Middle East and North Africa (MENA) region during recent global shocks like the COVID-19 pandemic, the 2022 wheat price spike, and the war in Gaza.

Methods

The authors treat Facebook reactions (Love, Care, Wow, Haha, Sad, Angry) as compositional data. Because these reactions are proportions that sum to 100%, an increase in one mechanically decreases the others. To handle this, they used a Centered Log-Ratio (CLR) transformation, mapping the proportions into Euclidean space where standard statistical methods apply.

Using k-means clustering on these transformed reactions, they identified four distinct "audience profiles":

  1. Solidarity: Dominated by "Love" reactions.
  2. Dismissal: Dominated by "Haha" reactions (often indicating sarcasm).
  3. Grief: Dominated by "Sad" reactions.
  4. Ambivalent: A flat profile with no dominant reaction.

They then tested whether a state-of-the-art Arabic-centric LLM (Fanar) and simple supervised classifiers could predict these profiles using only the post's text and metadata (like the country of the page and category).

Datasets

The study introduces the FoodReact-AR resource, which includes:

  • 74,241 Arabic Facebook posts (2020–2024) related to food security, clustered into the four profiles.
  • 6,225 reaction-blind LLM labels for zero-shot evaluation.
  • 800 posts with both reaction-informed LLM labels and independent native-speaker human annotations.
  • A 294-post matched ablation set to test if showing the LLM the reaction percentages actually helps its prediction.

Key Insights and Ideas

  • The "Haha" Signal: The "Dismissal" profile is the most elusive. It peaks during periods of economic stress (like the 2022 wheat shock) and is most common on posts from government organizations and TV channels. It represents a "derisive laughter" that text-only sentiment analysis usually misses.
  • LLMs are Too Literal: Zero-shot LLMs are surprisingly bad at "reading the room." While the LLM recovered "Grief" for 83.8% of posts, it caught "Dismissal" only 3.4% of the time. Even when the researchers showed the LLM that 60% of the audience reacted with "Haha," the model still refused to label the post as "Dismissal."
  • Derision is Predictable, but not from Tone: Simple supervised models (Logistic Regression) trained on reaction labels achieved 60% recall on Dismissal. These models didn't look for "sarcastic tone" but rather learned associations between specific actors (ministers, institutions) and the likely derisive response they provoke.
Figure 2: Per-profile recall of the zero-shot LLM and the page-disjoint supervised baseline on the same 6,225 posts. Training on reaction-derived labels recovers Dismissal, which the zero-shot model almost never predicts.

Key Results

  • Temporal Shifts: The study quantitatively confirmed that audience moods shifted with global events. Dismissal reactions doubled in 2022 during the wheat price shock, while Grief reactions peaked in 2024 during the Gaza food emergency.
  • Model Performance: The supervised baseline (text + metadata) significantly outperformed the zero-shot LLM, improving the macro-F1 score from 0.43 to 0.57.
  • Engagement: "Solidarity" posts are shared the most, while "Dismissal" posts generate the most comments—suggesting that derision drives discussion but not necessarily "travel" or virality in the same way as positive affiliation.
Table 1: The four reaction profiles and their mean reaction shares.

Limitations and Future Work

The authors acknowledge that their supervised baselines are lexical models that might be picking up on "outlet signatures" (e.g., knowing a certain page always gets mocked) rather than deep sarcasm. Additionally, the study used a single human annotator for the gold-standard set, which limits inter-annotator reliability checks.

Future work suggests that fine-tuning Arabic encoders (like MARBERT) or using few-shot prompting with reaction-derived examples could bridge the gap for LLMs. The authors have released their evaluation protocol to encourage the community to treat audience response as a distinct prediction target, separate from text sentiment.

Summary updated

Abstract

Facebook reactions record how an audience received a post, a signal absent from the post's text. We treat the six differentiated reactions on 74,241 Arabic food-insecurity posts from public Facebook pages (2020-2024) as compositions, apply a centered log-ratio transformation, and cluster them into four audience profiles: Solidarity (Love-dominant), Dismissal (Haha-dominant), Grief (Sad-dominant), and a flat Ambivalent profile. Dismissal peaks in 2022 (22.3%), the year of the wheat price shock, and grief in 2024 (14.0%), during the Gaza food emergency; a page-clustered GEE confirms both shifts (odds ratios 2.18 and 1.34), and they survive minimum-reaction filtering, alternative zero replacements and clustering methods, and bootstrap resampling. We then test whether the profiles can be predicted without reactions. A zero-shot Arabic LLM recovers Grief for 83.8% of posts but Dismissal for only 3.4%, and in a matched ablation, it shows the reaction percentages do not help. A simple classifier trained on the reaction-derived labels and tested on unseen pages recovers dismissal for 60.0% (macro-F1 0.56 against 0.43), mainly from mentions of officials and institutions. Derisive audience responses are thus learnable from text, yet a zero-shot LLM misses them. We release the corpus, LLM, and human annotation layers, and an evaluation protocol as FoodReact-AR.
BibTeX
@misc{zaghouani2026foodreact,
  author = {Wajdi Zaghouani and Mabrouka Bessghaier and Salma Azouzi and Shimaa Amer Ibrahim},
  title = {{FoodReact-AR: Learning Audience Reactions that Zero-Shot LLMs Miss in Arabic Food-Insecurity Discourse}},
  year = {2026},
  eprint = {2611.25682},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.25682},
}
Figure