TL;DR
This paper introduces a novel approach to improve voice search accuracy and speed for local business directory assistance by creating "geo-centric" language models. These models adapt to a user's specific location and local business density, leading to significant improvements over traditional nationwide or general local models.
What Problem Does This Paper Address?
Local Business Voice Search (LBVS) systems, like those used for commercial 411 services, aim to help users find businesses by name and location. While these systems are increasingly popular, especially on mobile devices, speech recognition accuracy for business names remains a significant challenge. This leads to user frustration, often requiring multiple attempts to get the correct information. Traditional approaches either use a single, massive nationwide language model (which is slow and less accurate for specific local needs) or separate, smaller local models (which can be inflexible if a user is near the edge of a predefined local area). The core problem is how to achieve high recognition accuracy and speed for local business searches without sacrificing user flexibility, especially on mobile devices where location awareness is readily available.
Methods
The paper proposes an algorithm for generating "geo-centric" language models (LMs). Instead of relying on broad geographical areas or a single nationwide model, these LMs are tailored to a user's immediate vicinity. The key innovation lies in how these geo-centric LMs are constructed:
- Adaptive Geo-Centricity: The algorithm determines the "local" area for an LM based on the density of businesses in a given region. This means the radius of an LM can vary – a smaller radius in a dense urban area like New York City, and a larger radius in a sparser area like rural Montana. This ensures that the LM covers businesses that are truly relevant to the user's immediate surroundings.
- Quantization of Business Locations: To manage the computational complexity of creating LMs for every possible user location, the algorithm "quantizes" business geo-coordinates. This means multiple businesses within a certain proximity are grouped together, and a single LM is generated for that group. This significantly reduces the number of LMs needed while still maintaining high relevance.
- Pre-compilation: The geo-centric LMs are pre-compiled. This allows for very fast recognition times at runtime, as the system doesn't need to build or adapt models on the fly.
- Combination Strategies: The paper also explores combining these geo-centric LMs with other models (like a "Top 2000" most frequent listings model or a national model) using techniques like "count merging" and "LM union" to further improve coverage and accuracy.
Datasets
The research uses data from an existing two-exchange directory assistance application. This dataset consists of 60,000 voice queries, each with a city/state and a business listing name. For evaluation, they specifically selected 15,000 queries that had a matching listing in their nationwide business database. Listings were categorized as "nationwide" if they appeared five or more times in query logs, and "local" otherwise.
Key Insights and Ideas
- Density is Key to "Local": The paper highlights that what constitutes a "local" listing isn't fixed but depends on the local business density. This is a crucial insight for tailoring LMs effectively.
- Quantization for Scalability: Grouping businesses based on their proximity and density is a clever way to make geo-centric LMs computationally feasible and scalable across a large country.
- Geo-centricity Boosts Accuracy and Speed: The core finding is that by focusing LMs on the user's immediate environment, both recognition accuracy and speed can be dramatically improved compared to broader models.
- Street Name Challenges: While geo-centric LMs excel at business names, they can sometimes struggle with queries containing street names, likely due to the limited coverage of specific street names in smaller, localized LMs.
Key Results
The experiments demonstrated significant improvements:
- Overall Accuracy: Geo-centric LMs achieved a 16.8% absolute improvement in overall sentence accuracy compared to a nationwide LM.
- Speed: Recognition time was 3-fold faster compared to a nationwide LM.
- Vs. Local Area LMs: Geo-centric LMs also outperformed general local area LMs, showing a 4.4% absolute increase in overall accuracy and a 5.5% increase in accuracy on local listings, while using less processing time.
- Street Name Queries: Interestingly, accuracy on queries containing street names decreased with geo-centric LMs, suggesting a trade-off that needs further investigation.
Limitations and Future Work
The authors acknowledge that the accuracy on street name queries decreased, which they attribute to the limited coverage of specific street names in their localized LMs. They suggest that more robust handling of street names in the LM construction process could mitigate this.
Future work includes:
- Optimizing the parameters used in the geo-centric LM generation and merging algorithms.
- Exploring the integration of language modeling with the search process itself.
- Investigating the impact of these language modeling approaches on the performance of trainable dialog managers.
Summary updated