TL;DR
KumbhDoot is a "similarity-first" architecture designed for massive, safety-critical events like the Kumbh Mela. By inverting the standard LLM-default design and treating generative models as a bounded fallback, the system achieves high scalability, offline resilience, and reduced hallucination risk for millions of users.
What Problem Does This Paper Address?
Building a conversational assistant for a mass gathering (supporting upwards of 80 million users) presents a unique "perfect storm" of constraints:
- Scale and Cost: Sending every query to a cloud-based Large Language Model (LLM) is prohibitively expensive at this volume.
- Connectivity: In dense crowds, mobile networks often fail, making cloud-only solutions unusable.
- Safety: Hallucinations regarding medical facilities or emergency routes can cause physical harm.
- Latency: Emergency queries require immediate, deterministic answers, not the slow, multi-step generation typical of LLMs.
The authors argue that the current industry trend—routing every user utterance to an LLM for understanding and generation—is poorly suited for these high-stakes, low-bandwidth environments.
Methods
KumbhDoot operates on the principle of Architectural Inversion. Instead of starting with a generative model, it uses an "escalation ladder" that prioritizes the cheapest and safest retrieval methods first.
The Escalation Ladder
As shown in the architecture, the system attempts to answer queries through three workload bands:
- Band A (Direct Retrieval): Uses a semantic cache to find previously approved Q&A pairs. This consumes zero generative tokens.
- Band B (Coded Orchestration): Handles multi-record requests (e.g., "List hospitals near me") using hand-coded logic and structured records, still without an LLM.
- Band C (LLM Synthesis): The LLM is invoked only for complex, multi-part synthesis or personalization, and its output is strictly constrained to retrieved, approved records.
Key Components
- InfoBins: The central data unit. Unlike raw text, an InfoBin is a structured object containing content, source authority, validity dates, and safety rules.
- Unified Semantic Cache: A single embedding-indexed store that performs five functions: knowledge base retrieval, intent classification, response caching, offline data bundling, and per-agent memory.
- Three-Tier Agent Model: A hand-coded (rather than framework-driven) orchestration layer that ensures decision paths remain inspectable and auditable.
Datasets
The system was developed using:
- Intent Exemplars: Approximately 520 labelled queries across 26 intent categories in English, Hindi, and Marathi.
- Curated Content: Official advisories, facility maps, and emergency contacts structured into InfoBins.
- Offline Bundle: A ~700 KB SQLite bundle containing FTS5 search indexes and compact embeddings for on-device use.
Key Insights and Ideas
- Similarity is a First-Class Citizen: The authors point out that the core operation of comparing vector representations (cosine similarity) predates generative AI by decades and is computationally cheap, deterministic, and auditable.
- LLM as a "Bounded Fallback": By treating the LLM as a secondary tool rather than the engine, the system mitigates hallucination. A "grounding-validation" step even strips out invented citations before the user sees them.
- Offline-First Design: By shipping a compact version of the semantic cache to the device, the assistant remains functional even when the network is completely down—a core requirement for crowd safety.
Key Results
- Token Elimination: The system demonstrates that LLM use can be eliminated in nearly all User Acceptance Testing (UAT) scenarios for common, repetitive queries.
- Cost Efficiency: The analytical cost model suggests that as the semantic cache "warms up" with common questions, the marginal cost per query approaches the cost of simple similarity computation, rather than the high cost of LLM tokens.
- Offline Capability: The architecture successfully supports hybrid retrieval (lexical + semantic) on-device with a minimal storage footprint.
Limitations and Future Work
The authors are transparent that this paper presents an architecture and analytical argument rather than a large-scale empirical study. Key areas for future work include:
- Empirical Validation: Measuring actual latency and cost speedups against a standard RAG baseline in a live production environment.
- Multilingual Accuracy: Testing how well cosine-based intent routing handles complex code-mixed speech (e.g., mixing Hindi and English) and regional dialects.
- Live Sync: Implementing real-time synchronization between the server's master index and the device's offline SQLite bundle to handle fast-changing advisories (like road closures).
- Emergency Escape Hatches: Integrating more robust non-AI paths, such as one-tap emergency buttons and direct SMS triggers.
Summary updated