Event-Triggered Ephemeral Group Communication and Coordination over Sound for Smart Consumer Devices

Doohwan Song, Ikjun Yeom, Honguk Woo

TL;DR

This paper introduces "tailtag," a novel framework for coordinating multiple smart devices that respond to voice commands. It uses inaudible sound for communication within a room to enable devices to spontaneously form "ephemeral groups" and elect a single responder, thus avoiding redundant reactions and improving user experience.

What Problem Does This Paper Address?

Modern smart devices, from smartphones to smart speakers, increasingly rely on voice interfaces. However, when multiple such devices are in close proximity and hear a wake-word (like "Hey Google" or "Hi Bixby"), they often all respond simultaneously, leading to a confusing and uncoordinated user experience. This "uncoordinated react" problem stems from the lack of a built-in mechanism for these devices to collaborate and decide which single device should handle the request. Existing solutions often rely on cloud infrastructure or complex user configurations, which aren't ideal for spontaneous, short-lived interactions. The paper aims to solve this by enabling devices to form temporary, ad-hoc groups and quickly elect a leader to provide a single, coordinated response.

Methods

The core of the proposed solution is the "tailtag" framework, which leverages two key ideas:

  1. Sound-based Communication: Devices use inaudible sound waves (18-21 kHz) to communicate. This is advantageous because sound is naturally confined to a room, effectively defining the boundaries of the ephemeral group without needing explicit network setup.
  2. Ephemeral Group Leader Election: When a hotword is detected, devices implicitly form an "ephemeral group." Within this group, they engage in a time-constrained leader election process. This process is inspired by bully algorithms but adapted for the unique constraints of sound communication and the need for rapid decision-making.

To improve the efficiency and success rate of this leader election, the paper introduces an adaptive messaging scheme:

  • Score-based Prioritization: Each device assigns an "application-specific score" to an event (e.g., based on the perceived distance of the hotword). This score determines the priority of a device's message.
  • Adaptive Delay: Devices adjust their transmission delay based on their score. Higher-scored messages are sent earlier, increasing their chances of being heard and suppressing lower-scored messages, thereby reducing collisions and unnecessary transmissions.
  • Predictive Ranking: An advanced version of the adaptive scheme uses a predictive model to estimate a device's rank within the group, further refining the scheduling and delay to improve consensus.

Datasets

Not explicitly discussed in terms of training datasets. The paper relies on experimental measurements and simulations. The "datasets" are essentially the results of experiments conducted using consumer devices (Samsung Galaxy S8/S6 smartphones) under various controlled conditions (distance, noise levels, group size, density, event dynamics).

Key Insights and Ideas

  • Sound as a Natural Group Boundary: Using inaudible sound for communication is a clever way to define the scope of an ephemeral group without relying on network infrastructure. It naturally limits communication to the immediate environment (e.g., a room).
  • Ephemeral Groups for Spontaneous Coordination: The concept of "ephemeral groups" is crucial. It allows devices to coordinate without prior setup, forming only when an event (like a hotword) occurs and dissolving afterward.
  • Adaptive Messaging for Low-Bandwidth, Time-Critical Communication: Sound communication has inherent limitations in data rate and reliability. The adaptive messaging scheme, with its score-based prioritization and dynamic delays, is essential for overcoming these limitations and achieving timely consensus.
  • Isomorphism of Voice Interfaces and Sound Communication: The paper highlights that voice interfaces (microphones and speakers) and sound communication share the same hardware and are often context-aware (room-based). This "isomorphism" simplifies the integration of the tailtag framework.

Key Results

  • High Accuracy: The tailtag framework, especially with the adaptive messaging scheme, achieved a high accuracy of 98% in successfully resolving a single, coordinated response within the given latency bound (1.1 seconds) for scenarios with five devices and slow event dynamics.
  • Reduced Non-Consensus Probability: The framework yielded an insignificant non-consensus probability, no more than 2%.
  • Improved Performance over Baselines: The adaptive algorithms (ad_D and ad_RD) consistently outperformed baseline algorithms (random delay and score-based delay) across various conditions (event dynamics, group size, spatial density).
  • Practical Integration: The prototype implementation demonstrated successful integration with existing voice assistant frameworks like Bixby.

Limitations and Future Work

  • Single Event Focus: The current framework is primarily designed for handling a single event and group type.
  • Variable Time Constraints: Future work aims to generalize the framework to support variable time constraints and different types of external events beyond hotwords.
  • Generalizing Group Structure: Extending the framework to support more complex group structures and communication patterns is also a direction for future research.
  • Score Function Extensibility: The paper notes that the score function is an example implementation and can be extended based on application requirements (e.g., giving priority to specific device types like home speakers).

Summary updated

Abstract

Voice-based interfaces have become one of the most popular device capabilities, recently being regarded as one flagship user experience of smart consumer devices. However, the lack of common coordination mechanisms might often degrade the user experience, especially when interacting with multiple voice-enabled devices located closely. For example, a hotword or wake-up utterance such as “hi Bixby” or “ok Google” frequently triggers redundant responses by several nearby smartphones. Motivated by the problem of uncoordinated react of voice-enabled devices especially in a multiple device environment, in this paper, we discuss the notion of an ephemeral group of consumer devices in which the member devices and the transient lifetime are implicitly determined by an external event (e.g., hotword detection) without any provisioned group structure, and specifically we concentrate on the time-constrained leader election process in such an ephemeral group. To do so: (i) We first present the sound-based multiple device communication framework, namely tailtag, that leverages the isomorphic capability of consumer devices for the tasks of processing hotword events and transmitting data over sound, and thus renders both the tasks confined to the same room area and enables the spontaneous leader election process in a unstructured group upon a hotword event. (ii) To improve the success rate of the leader election with a given time constraint, we then develop the adaptive messaging scheme especially tailored for sound-based data communication that inherently has low data rate. Our adaptive scheme utilizes an application-specific score that is individually calculated by a member device for each event detection, and employs score-based scheduling by which messages of a high score are scheduled first and so unnecessary message transmission can be suppressed during the election process. (iii) Through experiments, we also demonstrate that, when a hotword is detected by multiple smartphones in a room, the framework with the adaptive messaging scheme enables them to successfully achieve a coordinated response under the given latency bound, yielding an insignificant non-consensus probability, no more than 2%.
BibTeX
@inproceedings{song2019event,
  author = {Doohwan Song and Ikjun Yeom and Honguk Woo},
  title = {{Event-Triggered Ephemeral Group Communication and Coordination over Sound for Smart Consumer Devices}},
  booktitle = {Sensors},
  year = {2019},
  url = {https://openreview.net/forum?id=vqprxaiDGN},
}
Figure