Offline-First Quantized AI: Design and Evaluation of an On-Device Maternal-Health Assistant for Low-Connectivity Environments

Okenla AbdulQahhar Abimbola, Maurice Abraham John, Adepoju Jeremiah Moyondafoluwa, Omolaiye Samuel, Oko-Oba Sandra Chioma, Owoo Oluwamayowa Emmanuel, Thanni Adewuyi, Lily Aimas-Love, Benson Eniola, Abdulquddus Ajibade, Sylvia Essessien, Anyanwu Princess Emmanuella, Adereni AbiodunarXiv 2026 · arXiv

TL;DR

This paper details the development of an "offline-first" maternal health assistant for Android, designed to provide clinically validated medical advice in low-connectivity environments like rural Nigeria. By comparing quantized versions of Gemma and Qwen models, the researchers demonstrate that while ultra-compact models offer superior speed, slightly larger quantized models (like Gemma 4 E2B) are necessary to maintain the clinical safety and conversational context required for medical guidance.

What Problem Does This Paper Address?

In regions like Nigeria—which accounts for nearly 28% of global maternal deaths—access to timely medical information is a matter of life and death. While AI assistants could bridge this gap, most rely on cloud-based LLMs that require stable internet. In rural or low-resource settings, connectivity is often non-existent or prohibitively expensive.

The challenge is "compact local inference": how do you squeeze a capable medical AI onto a mid-range smartphone? The authors tackle the trade-off between clinical appropriateness (is the advice safe and correct?) and feasible execution (does the app crash or take minutes to respond?).

Methods

The authors developed an Android-based system that integrates:

  • On-Device LLMs: They compared two "ready-made" quantized configurations: Gemma 4 E2B (using Google’s LiteRT-LM) and Qwen3.5 0.8B (using llama.cpp).
  • Agentic RAG (Retrieval-Augmented Generation): Instead of the model just "guessing" from its weights, it can proactively decide to call a local tool to search a validated medical database stored on the phone (using SQLite FTS5/BM25).
  • Clinical Safety Boundary: A four-level framework (E0 to E3) that prioritizes safety, ensuring that "warning sign" symptoms trigger immediate escalation to human care rather than just educational information.
  • Deferred Synchronization: A system architecture that allows the app to function entirely offline and sync telemetry/logs only when a network connection becomes available.

Datasets

The study utilized several specialized datasets for evaluation:

  • Clinical Question Bank: A newly developed bank of 150 questions across nine domains (e.g., pregnancy milestones, nutrition, warning signs) reviewed by medical doctors.
  • Local Knowledge Package: A corpus of ~21,000 clinically validated passages derived from previous maternal health chatbot logs and official guidelines from the WHO and the Nigerian Ministry of Health.
  • Multi-turn Conversation Workload: 45 simulated conversations (312 turns) designed to test if the model remembers context (e.g., "I am 6 months pregnant") across an exchange.
  • Offline Agentic Benchmark: A set of tasks to test the model's ability to use tools, such as calculating a due date or looking up immunization schedules.

Key Insights and Ideas

  • The Quality-Resource Gap: The "aha moment" here is that model size isn't just about "intelligence"—it's about safety. While the tiny Qwen model (0.8B) was a "speed demon," it struggled significantly with context retention and safety. Gemma (4 E2B) was much heavier on resources but provided the "clinical weight" necessary for medical use.
  • RAG is not a Panacea: In a paired experiment, adding retrieved context improved 11 responses but actually degraded 4 others. In some cases, the model became "distracted" by the retrieved text, leading it to weaken the urgency of a medical emergency. This highlights that "more data" isn't always better if the model isn't large enough to reason about it correctly.
  • Hardware is the Bottleneck: The study found that while the app ran well on a 6GB RAM device, Gemma was killed by the OS on 4GB devices due to memory pressure. This defines a clear "hardware envelope" for deploying such tools in the field.

Key Results

  • Clinical Acceptance: 98% (147/150) of Gemma’s responses met the clinical acceptance criteria, with a mean quality score of 3.80/4.
  • Model Comparison: Gemma achieved a Quality Index of 82.7, significantly outperforming Qwen’s 52.4, particularly in safety and context retention.
  • Performance: On a physical 6GB Android handset, the median response latency was 14.3 seconds. While slower than a cloud API, it is considered "feasible" for an offline environment.
  • Retrieval Impact: In the RAG experiment, the system achieved a 14.9% improvement rate in responses, though 75.7% remained "neutral," meaning the model's internal knowledge was already sufficient for those specific queries.

Limitations and Future Work

  • Simulated vs. Real World: The evaluation used fixed benchmarks and simulated conversations. The authors acknowledge that live-user behavior and diverse linguistic patterns in the field might yield different results.
  • Hardware Sensitivity: The system is currently too heavy for "entry-level" (under 4GB RAM) smartphones, which are common in the target demographic.
  • Future Directions: The authors suggest optimizing CPU inference (e.g., thread scheduling and KV cache management) and conducting real-world pilot studies to assess user adoption and clinical outcomes. They also aim to refine the RAG "admission gates" to prevent the model from being distracted by irrelevant retrieved passages.

Summary updated

Abstract

Offline language models could extend access to maternal-health information where internet connectivity is unreliable, but deployment requires both clinically appropriate responses and feasible local execution. This study presents an offline Android assistant integrating ready-made quantized language models, a clinically validated local knowledge package, conversational context, and model-initiated local retrieval. Gemma 4 E2B and Qwen3.5 0.8B were compared across 45 conversations and 312 turns per configuration, including a dedicated offline agentic benchmark. Gemma achieved higher response-quality, context-retention, and safety scores; Qwen offered faster generation and lower memory use. Clinician assessment of 150 Gemma responses across nine clinical domains yielded a mean quality score of 3.80/4, with 147 responses meeting the acceptance criterion. A paired RAG-on/off experiment evaluated 74 questions: retrieval improved 11 responses, left 56 neutral, produced four adverse response changes, and left three pairs unassessable. RAG-on had descriptively higher mean usefulness, safety, correctness, and actionability in the conditional 70-pair subset excluding the four flagged pairs from both conditions. Physical-device testing demonstrated offline responses on a 6 GB Android handset, with a 14.3 s median latency, alongside successful reconnection synchronization in the integrated application. Together, the findings document clinical response quality and practical performance of the integrated offline maternal-health assistant.
BibTeX
@misc{abimbola2026offline,
  author = {Okenla AbdulQahhar Abimbola and Maurice Abraham John and Adepoju Jeremiah Moyondafoluwa and Omolaiye Samuel and Oko-Oba Sandra Chioma and Owoo Oluwamayowa Emmanuel and Thanni Adewuyi and Lily Aimas-Love and Benson Eniola and Abdulquddus Ajibade and Sylvia Essessien and Anyanwu Princess Emmanuella and Adereni Abiodun},
  title = {{Offline-First Quantized AI: Design and Evaluation of an On-Device Maternal-Health Assistant for Low-Connectivity Environments}},
  year = {2026},
  eprint = {2611.20012},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20012},
}
Figure