Mode preference in a simple data-retrieval task

Alexander I. Rudnicky

Introduction

In the early 1990s, human-computer interaction (HCI) research faced a conundrum regarding speech interfaces. While speech recognition technology promised a more natural way to interact with machines, empirical studies often labeled it a "loser" compared to traditional keyboard and mouse inputs. These assessments were typically based on raw efficiency metrics, such as total task completion time. This paper investigates a critical gap in this logic: if speech is objectively slower, why do users consistently express a preference for it?

Methods

The study utilized the Personal Information Database (PID), a component of the "OM" system, powered by the Sphinx recognition engine—a continuous speech, speaker-independent system. Subjects were tasked with retrieving the last four digits of a phone number for specific names in a database of conference attendees.

The research consisted of two primary experiments. Experiment 1 involved 9 subjects performing 20 trials in three modes (Speech, Keyboard, Scroller) to establish familiarity, followed by a "free choice" block. Experiment 2 extended this to 9 different subjects performing over 2,000 trials to observe long-term behavior and eliminate potential novelty effects. User actions were modeled using Finite State Machines (FSMs) to precisely measure "input time" (the time the user spends active) versus "transaction time" (the total time including system processing).

The Myth of the Novelty Effect

A common critique of early speech interface studies was that user preference might be driven by the novelty of the technology. However, longitudinal data from Experiment 2 suggests the opposite.

Figure 4: Proportion of trials by mode over 35 blocks of trials. Speech preference increases and stabilizes over time.

As shown in the figure above, the preference for speech actually strengthened as users became more experienced with the system, eventually asymptoting after approximately 250 utterances. In the second experiment, subjects chose speech 74.9% of the time, a figure that rose to 79.9% when difficult-to-pronounce names were excluded.

Input Effort vs. Transaction Time

The core insight of the study lies in the distinction between transaction time and input time. While early systems often had high "transaction times" due to slow processing and error correction, the "input time"—the physical and cognitive effort required from the user—was significantly lower for speech.

Figure 6: Comparison of mean transaction and input times (in seconds) for different input modes in Experiment 2.

The data indicates that users optimize for their own effort rather than the system's total clock time. Even when speech was slower in total transaction time (as seen in Experiment 1), the input time remained the lowest of all modes. By Experiment 2, with improved hardware, speech became competitive in total transaction time (9.46s) while maintaining a massive advantage in input efficiency (2.07s vs. 3.03s for keyboard and 4.39s for scroller).

Conclusion

This research demonstrates that efficiency-first metrics are insufficient for predicting user preference in multi-modal systems. Users are willing to tolerate system-induced delays if the interface reduces their personal "input effort." For designers, this suggests that the path to successful speech integration lies not just in faster recognition, but in minimizing the cognitive and physical load placed on the user during the interaction.

Summary updated

Abstract

No abstract available.
BibTeX
@inproceedings{rudnicky1993mode,
  author = {Alexander I. Rudnicky},
  title = {{Mode preference in a simple data-retrieval task}},
  booktitle = {INTERCHI Adjunct Proceedings},
  year = {1993},
  doi = {10.1145/259964.260078},
}
Figure