A comparison of input entry rates in a multimodal mobile application

Aleksi Melto, Markku Turunen, Jaakko Hakulinen, Anssi Kainulainen, Tomi HeimonenInterspeech 2008

TL;DR

This paper compares text and speech input methods for a mobile route guidance app, TravelMan. While speech input is theoretically fastest, domain-specific predictive text input showed significant learning effects and ultimately outperformed speech in some tasks due to network latency and recognition errors.

What Problem Does This Paper Address?

Entering information on mobile devices is notoriously challenging due to small screens and limited input methods. As mobile applications become more sophisticated, especially in domains like route guidance, efficient and user-friendly data entry becomes critical. This paper investigates how different input modalities – traditional multi-tap text, T9-style predictive text optimized for a specific domain (addresses), and speech input – compare in terms of speed and user experience within a real-world multimodal mobile application. The authors aim to provide empirical evidence on realistic input rates and user perceptions, moving beyond theoretical models.

Methods

The study employed a within-subjects design where 38 participants used the TravelMan application on a Nokia N95 smartphone. TravelMan is a multimodal route guidance app for public transport in Finland, featuring a "reel" metaphor for menu navigation. Participants performed four exercise tasks and 21 evaluation tasks. The input methods compared were:

  • Multi-tap text input: Standard character-by-character entry.
  • Predictive text input: A T9-style system optimized for the address domain, using a language model of 11,049 words (street names, places, numbers). This system integrated with the reel interface, narrowing down suggestions as the user typed.
  • Speech input: Utilizing a distributed, server-based Finnish speech recognizer.

Both objective metrics (input entry rates, task completion times, key presses, recognition accuracy) and subjective user experiences (via SERVQUAL questionnaires) were collected.

Datasets

The evaluation used a domain-specific language model for the predictive text input and speech recognition, consisting of 8,896 street names and addresses, 2,053 place names, and numbers from 1 to 100, totaling 11,049 words. This dataset was specific to the cities of Helsinki, Espoo, and Vantaa. No separate training datasets were discussed, as the focus was on user evaluation of existing input methods.

Key Insights and Ideas

  • Learning is Crucial for Predictive Text: The predictive text input method demonstrated a significant learning effect. Users became substantially faster and made far fewer errors (e.g., "clear key" presses) as they progressed through the tasks, eventually outperforming speech in later tasks.
  • Speech's Potential Hindered by Real-World Issues: While speech input was theoretically the fastest, its practical performance was hampered by network latency for audio transfer and speech recognition errors. These factors, combined with the need for retries, eroded its speed advantage.
  • Domain Optimization Matters: The domain-specific predictive text input, with its tailored language model and integration with the reel interface, proved to be a strong contender, suggesting that optimizing predictive text for specific application domains can yield excellent results.
  • Subjective vs. Objective Discrepancies: Users subjectively rated predictive text input very highly, often surpassing their expectations, even though speech was objectively faster in many scenarios. This highlights the importance of user perception and the impact of factors like error correction effort and interface design.

Key Results

  • Median Task Completion Times: Speech (13.7s), Predictive Text (17.2s), Multi-tap Text (30s).
  • Speech Recognition Rate: Overall 70.4%, with 97% of tasks completed successfully within three attempts.
  • Input Time (excluding transfer): Speech (5.7s), Predictive Text (15.6s), Multi-tap Text (28.8s).
  • Predictive Text Learning: Input entry rate increased from 5.35 wpm in the first task to 11.58 wpm in the last task. "Clear key" presses per character dropped from 0.41 to 0.02.
  • Subjective Ratings: Users found predictive text input and the TravelMan application superior to their expectations. Speech input was the least favored method.

Limitations and Future Work

  • Network Dependency: The study acknowledges that slow network connections significantly impacted speech input performance. Future work should investigate performance with faster, more reliable connections.
  • User Experience in Motion: The current study was conducted in a lab environment. The authors plan to conduct user experiments with participants on the move to better understand the efficiency of predictive text and speech input in dynamic contexts.
  • Speech Recognition Accuracy: While the study achieved reasonable results for real-world use, the speech recognition accuracy was lower than typical advertised rates. Further improvements in recognition accuracy could significantly boost speech input's practical advantage.
  • Domain Specificity: The findings are specific to the route guidance domain. Further research is needed to generalize these findings to other mobile application domains.

Summary updated

BibTeX
@inproceedings{melto2008comparison,
  author = {Aleksi Melto and Markku Turunen and Jaakko Hakulinen and Anssi Kainulainen and Tomi Heimonen},
  title = {{A comparison of input entry rates in a multimodal mobile application}},
  booktitle = {Interspeech},
  year = {2008},
}
Figure