Say what? why users choose to speak their web queries

Maryam Kamvar, Doug BeefermanInterspeech 2010

TL;DR

This paper investigates why users opt to speak their web search queries instead of typing them on mobile devices. They found that factors like using an unconventional keyboard, searching for location-based information, or needing a "hands-free" experience strongly predict spoken queries, while surprisingly, longer queries are more likely to be typed.

What Problem Does This Paper Address?

As voice search capabilities become more integrated into mobile applications, understanding user behavior is crucial for designing effective interfaces and improving speech recognition systems. The authors aim to identify the contextual factors that influence a user's decision to speak a web search query versus typing it. This knowledge can help optimize user interfaces, making speech input more prominent when beneficial and less so when it's likely to be a hindrance. Furthermore, this understanding can inform speech recognition models by providing insights into query characteristics that are more amenable to spoken input.

Methods

The researchers analyzed large-scale logs from the Google Mobile Application (GMA), which offers users a choice between typing and speaking their search queries. They employed a data-driven approach, examining correlations between various query attributes and the chosen input modality. Their analysis involved classifying queries based on keyboard type, query length, query popularity, and query topic. They also analyzed the type of search results returned, specifically focusing on "quick results" that can be consumed without further interaction.

Datasets

The study utilized a dataset of queries issued by users on BlackBerry phones in the United States over a 4-week period during the summer of 2009. The dataset was specifically curated to include users who had issued at least one spoken and one typed query, ensuring they were aware of both input modalities. Approximately 75,000 users and over 1,000,000 queries were sampled from this user base.

Key Insights and Ideas

  • Unconventional Keyboards Favor Speech: Users are significantly more likely to speak queries when using a compressed keyboard, suggesting that input difficulty is a major driver for choosing voice.
  • Location and "Hands-Free" are Key: Queries related to a user's location (e.g., local businesses, weather) or those that can be answered in a "hands-free" manner (e.g., quick results like sports scores or maps) are strong predictors of spoken queries. This highlights the utility of voice in specific, often on-the-go, scenarios.
  • The Counter-Intuitive Length Effect: Contrary to what one might expect, longer queries are more likely to be typed than spoken. This suggests that the cognitive load or "articulatory buffer" limitations for spoken queries become more pronounced with increasing length.
  • Query Topic Matters: Certain categories, like "Local," "Food & Drink," and "Travel," show a higher propensity for spoken queries, aligning with the location-based and situational context findings. Conversely, "sensitive" topics (e.g., "Adult," "Lifestyles") are more likely to be typed, indicating a desire for privacy.
  • "Quick Results" Drive Hands-Free Preference: Queries that yield "quick results" (information displayed directly on the search results page) are more frequently spoken, reinforcing the "hands-free" hypothesis.

Key Results

  • Users with compressed keyboards are 20% more likely to issue a spoken query compared to those with full QWERTY keyboards.
  • Queries related to location or situational context (e.g., Local, Food & Drink, Travel categories) show a higher probability of being spoken.
  • The probability of speaking a query shows an inflection point at 6 words; queries shorter than 6 words are more likely to be spoken, while longer queries are increasingly typed.
  • Spoken queries trigger "quick results" 12% more often than typed queries.
  • Queries returning "Maps" quick results are spoken approximately 50% of the time.
  • Conversely, "Spelling" results (e.g., "did you mean...") are typed 90% of the time, as the speech recognition backend handles spelling correction.

Limitations and Future Work

The authors acknowledge that their study is based on a specific mobile platform (BlackBerry) and a particular time frame. Future work could explore these patterns across a wider range of devices and user demographics. They suggest several avenues for improvement:

  • Enhance Local Query Handling: Further improvements in accuracy and result presentation for local queries could boost voice search adoption.
  • Enable Streaming Input: Allowing for streaming speech input and real-time recognition feedback could reduce cognitive load and make longer spoken queries more feasible.
  • Optimize "Hands-Free" Results: Formatting result pages to be more "hands-free" friendly, especially for spoken queries, could enhance the user experience.

Summary updated

BibTeX
@inproceedings{kamvar2010say,
  author = {Maryam Kamvar and Doug Beeferman},
  title = {{Say what? why users choose to speak their web queries}},
  booktitle = {Interspeech},
  year = {2010},
}

Loading related content...

Figure