ArSciKids: Arabic Curriculum-Aligned Datasets for Scientific Experiment and Multiple-Choice Question Generation in Primary Education

Taghrid Housari, Massa Rankoussi, Jafar Abdul Majeed, Wael Tarabishi, Khloud Al Jallad, Nada GhneimarXiv 2026 · arXiv

TL;DR

The authors introduce ArSciKids, the first collection of manually curated, structured Arabic datasets specifically designed for curriculum-aligned science education in grades 1–6. By fine-tuning a variety of open-source LLMs, the study demonstrates that while 4B-parameter models show strong potential for generating experiments and assessments, traditional metrics like ROUGE and BLEU often fail to capture the scientific accuracy and pedagogical value required for the classroom.

What Problem Does This Paper Address?

While Arabic is a major global language, its footprint in educational Natural Language Processing (NLP) is disproportionately small. Most existing Arabic datasets are either machine-translated from English or lack the structural depth needed for "generation-ready" tasks.

Teachers need AI tools that can generate lesson-specific experiments or multiple-choice questions (MCQs) that adhere to national standards. However, without high-quality, curriculum-aligned data, models struggle with localized terminology, grade-appropriate linguistic complexity, and the logical flow of scientific procedures. ArSciKids aims to bridge this gap by providing authentic, structured data derived directly from primary school textbooks in Saudi Arabia, Syria, and Jordan.

Methods

The researchers employed a rigorous manual curation process, moving away from synthetic or translated data. They extracted content from 18 PDF textbooks and educational platforms to build two distinct datasets.

To test the utility of these datasets, they fine-tuned ten open-source models—ranging from sub-billion to 4-billion parameters (including Gemma-3-4B, Qwen1.5, and Falcon3) — using LoRA (Low-Rank Adaptation). The evaluation was two-pronged:

  1. Automatic Metrics: ROUGE and BLEU for lexical similarity, and letter accuracy for MCQ answers.
  2. Human Evaluation: A five-criterion protocol assessing factual accuracy, structural integrity, logical coherence, practical value, and language fluency.

Datasets

The paper introduces two primary resources:

  • ArSciKids_Exp: 999 records of scientific experiments. Each record is structured into fields: title, objective, tools, steps, observations, conclusion, keywords, and grade level.
  • ArSciKids_QA: 2,167 multiple-choice questions. These include lesson references, the question text, four options, the correct answer, and a detailed explanation.

As shown in the domain distribution below, the datasets heavily favor Biology, reflecting the standard primary science curriculum.

Figure 6: Scientific Experiment & MCQs Datasets Domain Distribution

Key Insights and Ideas

  • The "Lexical vs. Logical" Gap: A major "aha moment" was the discrepancy between automatic metrics and human judgment. A model might achieve a high ROUGE score (indicating it used the right words) but fail human evaluation because the experiment it generated was logically incoherent or physically impossible to perform.
  • Normalization is Not a Silver Bullet: The authors tested basic text normalization (standardizing characters). Surprisingly, the effect was highly model-dependent. It significantly boosted Gemma-2-2B but caused a severe performance degradation in Qwen1.5-1.8B for certain grade levels.
  • Model Scale Matters: Models with fewer than 2 billion parameters consistently struggled with the complex reasoning required for experiment generation. The 4B-parameter models (Gemma-3 and MedGemma) emerged as the "sweet spot" for handling Arabic scientific content effectively.

Key Results

  • Top Performer: Gemma-3-4B led the pack, achieving a ROUGE-1 of 0.9664 on experiment generation for grades 1–2 and reaching 100% answer accuracy in the MCQ generation task under specific settings.
  • MCQ Quality: Human evaluation showed that Gemma-3-4B and MedGemma-4b maintained high structural quality, with scores consistently above 96% across most criteria.
  • Experiment Coherence: While models were great at "Structural Integrity" (getting all the sections right), they struggled with "Logical Coherence" and "Factual Accuracy," where scores for the best models hovered around 60-70%.

Below is a sample of the structured MCQ output the models were trained to produce:

Figure 4: MCQ Dataset Samples

Limitations and Future Work

The authors acknowledge that while ArSciKids is a significant step forward, it remains a "low-resource" dataset in the context of modern LLM training. The 999 experiment records, while high-quality, are limited in volume.

Future work will focus on:

  • Expanding Scope: Moving beyond primary science into other subjects and higher grade levels.
  • Logical Reasoning: Developing better interventions to ensure that the "steps" of an experiment actually lead to the "conclusion" provided.
  • Human-in-the-loop: Strengthening the human evaluation protocol to better guide model fine-tuning toward pedagogical usefulness rather than just word-matching.

Summary updated

Abstract

Despite growing interest in Arabic natural language processing, curriculum-aligned educational resources remain scarce compared with English, and few public datasets for Arabic curriculum content with structured, generation-ready annotations are available, particularly for primary-school science. We introduce ArSciKids, a collection of two Arabic datasets designed for curriculum-aligned educational content generation: (1) a scientific-experiments dataset containing 999 manually curated records from official primary-school science textbooks in Saudi Arabia, Syria, and Jordan, and (2) an assessment-question dataset containing 2,167 multiple-choice questions collected from the Saudi Manhaji educational platform. The experiment records are structured with titles, objectives, tools, procedures, observations, conclusions, keywords, and grade levels, while the assessment records contain lesson references, questions, four answer options, correct answers, and explanations. Both datasets cover grades 1-6 and consist of real-life Arabic educational content rather than machine-translated or synthetically generated material. We assess the usefulness of these resources for generation by fine-tuning ten open models spanning sub-billion to several-billion parameter scales using LoRA. Models are evaluated using automatic metrics, including ROUGE and BLEU, and answer accuracy for multiple-choice generation. Moreover, we evaluate the results using a five-criterion human evaluation protocol. The results show substantial differences across model scales and tasks. ArSciKids will be publicly available, and we hope that it provides a manually curated, structured foundation for research on Arabic educational NLP, curriculum-aware generation, and low-resource language modeling.
BibTeX
@misc{housari2026arscikids,
  author = {Taghrid Housari and Massa Rankoussi and Jafar Abdul Majeed and Wael Tarabishi and Khloud Al Jallad and Nada Ghneim},
  title = {{ArSciKids: Arabic Curriculum-Aligned Datasets for Scientific Experiment and Multiple-Choice Question Generation in Primary Education}},
  year = {2026},
  eprint = {2611.01191},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.01191},
}
Figure