Hybrid Retrieval for Government Scheme Search in English and Marathi

Aishwarya Gunjal, Kanchan Khemnar, Pawan Poojary, Pranav Kulkarni, Raviraj Joshi, Manish KhodaskararXiv 2026 · arXiv

TL;DR

This paper introduces a bilingual search system for Maharashtra government schemes that bridges the gap between informal citizen queries and formal administrative language. By fusing traditional lexical search (BM25) with modern semantic embeddings (MahaSBERT and IndicSBERT), the authors demonstrate that a hybrid approach achieves superior precision and ranking quality over either method alone, particularly in a bilingual English-Marathi context.

What Problem Does This Paper Address?

Navigating government services is often a nightmare of "vocabulary mismatch." A citizen might search for "financial help for farmers," while the official document is titled "Provision of Agricultural Credit." Furthermore, in states like Maharashtra, information may be authored in English but searched for in Marathi (or vice versa).

Standard search engines often struggle with these two hurdles:

  1. Semantic Gap: Exact keyword matching fails when users use paraphrases instead of formal terminology.
  2. Multilingual Barriers: Relevant documents might exist in a different language than the query.

The authors tackle these challenges by building and evaluating a retrieval pipeline specifically tuned for public service discovery in a bilingual environment.

Methods

The researchers developed a hybrid retrieval framework that operates in two parallel streams:

  1. Lexical Retrieval (BM25): Using Elasticsearch, they implemented a field-weighted BM25 algorithm. They assigned higher weights to titles and categories (3x and 2x respectively) compared to the main description to ensure that specific scheme names remain highly discriminative.
  2. Semantic Retrieval (Dense): They tested three configurations to generate vector representations of queries and documents:
  3. IndicSBERT: A multilingual model covering several Indian languages.
  4. MahaSBERT: A model specifically fine-tuned for Marathi.
  5. Translation-mediated MiniLM: A baseline that translates queries to English before using a standard English encoder.
  6. Score Fusion: The system combines the normalized scores from both streams using an equal-weight convex combination ($\alpha = 0.5$). This "hybrid" score determines the final ranking.
Figure 1: Core retrieval pipeline. Offline collection and alignment support lexical and semantic indexes. Query-time rankings are combined through normalized score fusion; the same scheme identifiers link both language versions.

Datasets

The study utilizes a specialized, manually curated collection:

  • The Corpus: 64 distinct Maharashtra government schemes and services, aligned in both English and Marathi.
  • The Queries: 150 manually curated bilingual queries (75 English, 75 Marathi) mapped to relevant schemes.
  • Alignment: English records were translated to Marathi using the Google Translation API, followed by domain-specific glossary checks and native-speaker reviews.
Statistics of the aligned English-Marathi collection.

Key Insights and Ideas

  • BM25 is a "Goliath": In specialized administrative domains, lexical matching is incredibly strong. Because scheme names and categories are unique and informative, BM25 alone achieved a high Mean Reciprocal Rank (MRR) of 0.864.
  • Dense Retrieval is Not a Silver Bullet: On its own, semantic search performed significantly worse than BM25 (MRR ~0.66). This suggests that for government schemes, "meaning" alone isn't enough; the specific "keywords" of the law or department still matter immensely.
  • Complementarity Wins: The "aha moment" is that while dense retrieval is weaker alone, it catches the paraphrases that BM25 misses. Fusing them provides a "best of both worlds" effect, pushing the MRR to 0.879.
  • Language-Specific vs. Multilingual: Interestingly, the Marathi-focused MahaSBERT yielded the highest precision and MRR in the hybrid setup, while the multilingual IndicSBERT provided the best overall recall. This highlights a trade-off between depth (precision) and breadth (recall) when choosing encoders.

Key Results

The hybrid approach consistently outperformed the baselines across major metrics:

  • Ranking Quality: The MahaSBERT-hybrid system achieved the highest MRR of 0.879, a modest but statistically relevant improvement over the 0.864 BM25 baseline.
  • Precision: Hybrid fusion increased precision from 0.781 (BM25) to 0.808 (MahaSBERT-hybrid).
  • Recall: The IndicSBERT-hybrid configuration reached the highest recall at 0.950.
  • Relative Gain: While the jump from BM25 to Hybrid is modest (~1.7% MRR gain), the jump from Dense-only to Hybrid is massive (~34% relative improvement), proving that lexical signals are essential anchors for semantic models in this domain.
Comparison of retrieval results across lexical, dense, and hybrid methods.

Limitations and Future Work

The authors identify several areas for growth:

  • Scale: The current evaluation is limited to 64 schemes. Real-world government portals contain thousands, which may change the dynamics of "exact match" vs. "semantic" signals.
  • Static Weights: The 50/50 split between lexical and semantic scores is arbitrary. Future work could explore query-dependent weights (e.g., giving more weight to BM25 for queries containing proper nouns).
  • Eligibility: The system currently finds relevant schemes but does not determine if a user is eligible. Integrating structured eligibility logic with unstructured text retrieval remains a significant open challenge.
  • Translation Nuance: Administrative Marathi is complex; the authors note that general-purpose sentence embeddings may still struggle with highly technical legal terminology.

Summary updated

Abstract

Retrieving government schemes requires matching citizens' information needs to formal administrative descriptions, often across languages and terminology. We present a bilingual search system for Maharashtra government schemes and public services that combines BM25 retrieval with sentence-embedding similarity through equal-weight score fusion. The system indexes aligned English-Marathi scheme descriptions and supports multilingual, Marathi-focused, and translation-mediated semantic configurations. Evaluation on manually curated bilingual queries compares BM25, dense retrieval, and hybrid retrieval using IndicSBERT and MahaSBERT. BM25 provides a strong baseline with a mean reciprocal rank (MRR) of 0.864. Hybrid retrieval increases MRR to 0.872 with IndicSBERT and 0.879 with MahaSBERT, whereas the corresponding dense-only systems achieve 0.663 and 0.655. MahaSBERT-based fusion also increases precision from 0.781 to 0.808 while matching BM25 recall at 0.946. These results support the complementary use of lexical and semantic signals in this collection, while showing that improvements over a strong lexical baseline are modest and metric-dependent. The study provides an application-focused analysis of hybrid retrieval for bilingual public-service information access.
BibTeX
@misc{gunjal2026hybrid,
  author = {Aishwarya Gunjal and Kanchan Khemnar and Pawan Poojary and Pranav Kulkarni and Raviraj Joshi and Manish Khodaskar},
  title = {{Hybrid Retrieval for Government Scheme Search in English and Marathi}},
  year = {2026},
  eprint = {2611.26514},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.26514},
}
Figure