TL;DR
MITRA-Swarm is a multi-agent architecture that improves Classical Sanskrit translation by explicitly modeling linguistic features like word fusion (Sandhi) and omitted subjects (ellipsis). By decomposing translation into specialized stages and employing a rigorous two-judge evaluation system, the researchers achieved statistically robust gains over monolithic foundation models, proving that explicit linguistic structure still provides value in the era of massive LLMs.
What Problem Does This Paper Address?
Classical Sanskrit is a "boss level" challenge for Machine Translation (MT). Unlike the high-resource, concatenative languages (like English) that foundation models are primarily trained on, Sanskrit features three systematic hurdles:
- Sandhi (Morphophonemic Fusion): Words merge at their boundaries (e.g., pustakam + asti becomes pustakamasti), which confuses standard tokenizers and leads to dictionary-lookup failures.
- Samāsa (Nominal Compounding): Dense compounds can span entire clauses, leaving the relationships between parts implicit.
- High Ellipsis (Zero Anaphora): Because of strict poetic meters, Sanskrit frequently omits subjects, objects, or even verbs.
Monolithic models often struggle to resolve these in a single generation call, leading to "hallucinated" subjects or semantic drift. Furthermore, Sanskrit is a low-resource language with limited parallel corpora, making it difficult for models to learn these nuances through scale alone.
Methods
The authors propose MITRA-Swarm, a pipeline that replaces the single "black box" generation call with a modular multi-agent system:
- Sandhi Parser Agent: Decomposes phonetic mergers and compounds into a structured morphosemantic guide before translation begins.
- Parallel Translator Array: Drafts are generated concurrently by a generalist frontier model (gemma-4-26B-A4B-it) and a domain-specialized fine-tune (gemma-2-mitra-it).
- Ellipsis Restorer with Voice Guarding: This agent looks at surrounding context to restore missing arguments. Crucially, it uses "voice guarding" to avoid inventing active-voice subjects in impersonal legal (Dharmaśāstra) or philosophical (Sūtra) texts, where a passive or impersonal tone is grammatically intentional.
- Two-Judge Architecture: To ensure unbiased evaluation, the system uses two separate judges. A "routing judge" ($J_{route}$) from the same model family as the generators handles internal critique loops. However, the final reported scores come from an independent "reporting judge" ($J_{report}$, GPT-4o-mini) with no shared vendor lineage to prevent "self-preference" bias.
Datasets
The study utilized the MitraSamgraha corpus, a comprehensive dataset for Classical Sanskrit. Evaluation was performed on a stratified sample of 1,000 verses across five distinct registers: epic, sutra, vedic, kavya, and legal. To prevent data leakage, the authors screened a 192,151-entry retrieval index, verifying zero reference leaks during evaluation.
Key Insights and Ideas
- Discourse Matters More Than Scale: The single largest individually significant improvement came from the Ellipsis Restorer (+0.44 MQM). This suggests that for complex languages, a linguistically motivated discourse module is more effective than simply using a larger model or generic self-critique.
- The Self-Preference Bias is Real: The authors quantified a significant bias in LLM-as-a-judge frameworks. The routing judge ($J_{route}$) consistently scored its own model family 2–7 MQM higher than the independent reporting judge ($J_{report}$). This highlights the necessity of "vendor-independent" evaluation in MT research.
- Register-Awareness is Critical: In Sanskrit, "correcting" an ellipsis by adding a subject can actually be a mistake if the genre demands an impersonal voice. The "voice guard" mechanism demonstrates that MT systems need to understand the intent of a genre, not just the grammar of a sentence.
Key Results
The researchers used the Multidimensional Quality Metrics (MQM) framework for evaluation. While the absolute gains appear small, they are statistically significant on a metric that saturates near 85-86:
- Full Swarm vs. Baseline: The swarm reached 85.90 MQM, beating the monolithic baseline of 85.27 ($\Delta = +0.63$, $p_{adj} = 0.0018$).
- Failure Reduction: Severe per-verse failures (MQM < 80) dropped from 9.0% to 7.6%.
- Efficiency Trade-off: These gains come at a cost; the full swarm requires roughly 9.2× the inference tokens compared to the baseline (see Figure 3).
Limitations and Future Work
- Small Absolute Effects: The authors note that the effects are concentrated in the "low tail" (fixing the worst failures) rather than shifting the entire distribution.
- Compute Cost: The multi-agent approach is significantly more expensive than single-pass models, which may limit real-time deployment without further optimization.
- Single Point of Failure: While the two-judge system is robust, the final metric still relies on a single reporting model. Future work aims to use a multi-judge panel.
- Expansion: The team plans to extend this architecture to other classical languages like Tibetan and Pāli.
Summary updated