Design and Implementation of a Disambiguity Framework for Smart Voice Controlled Devices

Kehua Lei, Tianyi Ma, Jia Jia, Cunjun Zhang, Zhihan YangIJCAI 2019 · OpenReview

Abstract

With about 100 million people using it recently, SVCD (Smart Voice Controlled Device) are becoming demotic. Whether at home or in an office, usually, multiple appliances are under the control of a single SVCD and several people may manipulate an SVCD simultaneously. However, present SVCD fails to handle them appropriately. In this paper, we propose a novel framework for SVCD to eliminate orders’ ambiguity for single user or multi-user. We also design an algorithm combining Word2Vec and emotion detection for the device to wipe off ambiguity. Finally, we apply our framework into a virtual smart home scene and the performance of it indicates that our strategy resolves the problems commendably.

Introduction

Despite the massive adoption of Smart Voice Controlled Devices (SVCDs) like Amazon Echo or Google Home, they often struggle in complex, real-world scenarios. Current devices typically fail in two major areas: semantic ambiguity in multi-device environments (e.g., "turn on the light" when multiple lights exist) and multi-user conflicts where simultaneous commands lead to execution errors or user frustration. This paper introduces a framework designed to resolve these ambiguities through semantic analysis, emotional context, and user habits.

Methodology

The proposed framework employs a multi-stage approach to process audio commands and resolve conflicts.

The proposed disambiguity framework for single and multi-user scenarios.

Voice Separation and Priority

The system first uncouples overlapping acoustic commands from different users. Priorities are assigned based on acoustic features and the degree of ambiguity. If commands conflict, the system acts as a mediator to reach a consensus; otherwise, it executes them concurrently.

Semantic Matching and Contextual Inference

To resolve vague commands, the system utilizes a Word2Vec network to convert spoken instructions and default device commands into semantic vectors. The Euclidean distance between these vectors identifies the closest match. If ambiguity persists, the system integrates a multi-path neural network for emotion detection and historical user habit data to infer intent. As a final fallback, the system engages in a multi-round dialogue to seek clarification from the user.

Experimental Evaluation

The framework was evaluated in a 2.5D virtual smart home environment, allowing for controlled testing of multi-device interactions.

The 2.5D virtual smart home environment used for system evaluation.

A user study involving 10 participants compared the proposed system against a commercial baseline (XiaoMi smart speaker). Participants controlled virtual lights under various ambiguous scenarios. The proposed SVCD achieved an average satisfaction score of 7.14 out of 10, significantly outperforming the baseline score of 5.0. These results highlight the effectiveness of layering acoustic emotion detection and habit tracking over standard semantic matching.

Conclusion

This research demonstrates that multi-modal disambiguation—combining text semantics, acoustic emotion, and historical context—provides a more robust and human-like interaction for smart home devices. While the current study is limited by a small sample size and a virtual environment, it establishes a clear path for future deployment in physical smart homes with expanded device support.

Summary updated

Abstract

With about 100 million people using it recently, SVCD(Smart Voice Controlled Device) are becoming demotic. Whether at home or in an office, usually, multiple appliances are under the control of a single SVCD and several people may manipulate an SVCD simultaneously. However, present SVCD fails to handle them appropriately. In this paper, we propose a novel framework for SVCD to eliminate orders’ ambiguity for single user or multi-user. We also design an algorithm combining Word2Vec and emotion detection for the device to wipe off ambiguity. Finally, we apply our framework into a virtual smart home scene and the performance of it indicates that our strategy resolves the problems commendably.
BibTeX
@inproceedings{lei2019design,
  author = {Kehua Lei and Tianyi Ma and Jia Jia and Cunjun Zhang and Zhihan Yang},
  title = {{Design and Implementation of a Disambiguity Framework for Smart Voice Controlled Devices}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2019},
  url = {https://openreview.net/forum?id=N7HxBimtfy},
}
Figure