SadaqahStories: A Verified English-Arabic Corpus of Charity Stories for Children, with an Audit of Errors in AI-Drafted Religious Narratives

Wajdi Zaghouani, Spencer StrikerarXiv 2026 · arXiv

TL;DR

The SadaqahStories project introduces a verified, parallel English-Arabic corpus of 41 children's stories about Islamic charity. By releasing the original AI-generated drafts alongside audited and final versions, the authors provide a unique dataset for studying Large Language Model (LLM) hallucinations in sensitive religious and historical contexts.

What Problem Does This Paper Address?

Generative AI makes content creation cheap, but it often prioritizes "sounding right" over being factually accurate. In the domain of religious education, where accuracy is paramount, LLMs frequently "hallucinate" details—inventing hadith wordings, historical dates, or specific numbers (like the exact count of camels in a relief effort) that have no basis in source texts.

The authors address the scarcity of high-quality, bilingual (English-Arabic) digital material for children on Islamic giving (Sadaqah, Zakat, and Waqf). More importantly, they tackle the challenge of auditing AI-generated narratives to ensure they are safe and accurate for educational use.

Methods

The researchers followed a rigorous five-stage pipeline to build the corpus:

  1. Drafting (V0): 62 bilingual story drafts were generated using LLM assistance.
  2. Selection: Filtering out near-duplicates and stories that lacked verifiable sources or might be religiously contested.
  3. Audit (V1): A line-by-line check against canonical sources (Qur'an, hadith collections like sunnah.com, and historical records). Every correction was logged and categorized.
  4. Child-oriented Rewrite (V2): Simplifying the audited text for readers aged 9–12 without adding new facts.
  5. Scholarly Sign-off: A final review by an Islamic scholar to ensure religious and linguistic integrity.

The team also developed a specialized correction log using a custom error typology to track exactly how the AI-drafted narratives deviated from the truth.

Datasets

The primary output is the SadaqahStories Corpus, which includes:

  • 41 stories (328 page-level passages) across three versions: AI-assisted draft (V0), audited (V1), and final (V2).
  • A per-story correction log with error codes.
  • 58 source links to canonical religious and historical texts.
  • Charity Lexicon: An 81-entry dataset of charitable terms in Modern Standard Arabic and Qatari Arabic, normalized for NLP tasks.
The distribution of stories by theme and the type of evidence supporting them.

Key Insights and Ideas

  • Vividness vs. Veracity: LLMs often hallucinate specific, concrete details (prices, names, staff counts) to make a story more engaging. While this works for fiction, it is dangerous for historical and religious narratives.
  • Scripture vs. Narrative: While LLMs sometimes misquote scripture, the majority of errors occurred in the historical frame surrounding the scripture—the "who, when, and how much" of the story.
  • Language-Specific Hallucinations: The Arabic drafts were not simple translations of the English ones and contained their own unique errors. For example, one Arabic draft incorrectly labeled voluntary charity as Zakat (obligatory alms), a distinction the English version got right.
  • The Cost of Correction: Auditing and correcting AI drafts often involves significant "deletion." The audited versions were 38–48% shorter than the AI drafts because so much unsupported "fluff" had to be removed.

Key Results

  • High Error Rate: 39 out of the 41 published drafts (95%) contained at least one substantive error.
  • Error Typology: The most frequent errors were "Unsupported specific details" (16 stories) and "Fabricated entities/quotations" (9 stories).
  • Readability Improvement: The rewriting process successfully lowered the English Flesch-Kincaid grade level from a difficult 14.0 (college level) to a child-friendly 6.1.
  • Platform: The corpus is served via a free, account-free website designed to protect minor privacy while providing interactive educational tools.
The SadaqahStories platform showing the bilingual interface and evidence labels.

Limitations and Future Work

The authors acknowledge that the corpus is relatively small (41 stories). The error coding was performed by a single coder, meaning inter-annotator agreement wasn't measured. Additionally, while English readability was measured using standard formulas, a specialized Arabic readability model (like BAREC) is needed for future iterations.

Future work includes measuring how children actually interact with these stories in classroom settings and expanding the corpus by auditing the remaining drafts that didn't make the initial cut.

Summary updated

Abstract

We present SadaqahStories, a parallel English-Arabic corpus of 41 illustrated four-page stories about charitable giving in Islam (Sadaqah, Zakat and Waqf), written for readers aged 7 to 18. The stories were first drafted with large language model (LLM) assistance, then audited against hadith collections, Qur'an references and historical sources, and finally rewritten for children and signed off by an Islamic scholar. Every story carries an evidence label and links to its sources. We release all three versions of each story (AI-assisted draft, audited text, final child-oriented text) together with the correction log, so the corpus can be used to study how AI drafts of religious and historical content go wrong. The audit found substantive errors in 39 of the 41 drafts. The most frequent were unsupported specific details (dates, figures, prices, names), invented quotations, institutions or anecdotes, and conflation of separate events or people; six drafts contained errors in the wording, grading or ruling of a religious text. Rewriting cut the estimated Flesch-Kincaid grade of the English text from 14.0 to 6.1. We also release an 81-entry charity lexicon covering Modern Standard Arabic and Qatari Arabic, and describe the free, account-free website that serves the corpus (https://sadaqahjourney.com).
BibTeX
@misc{zaghouani2026sadaqahstories,
  author = {Wajdi Zaghouani and Spencer Striker},
  title = {{SadaqahStories: A Verified English-Arabic Corpus of Charity Stories for Children, with an Audit of Errors in AI-Drafted Religious Narratives}},
  year = {2026},
  eprint = {2611.12078},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.12078},
}
Figure