IMPAct: A Holistic Framework for Mixed Reality Robotic User Interface Classification and Design

Dennis Krupke, Jianwei Zhang, Frank SteinickeMultimodal Technol. Interact 2019 · DOI

TL;DR

This paper introduces the IMPAct framework, a novel taxonomy for classifying and designing mixed reality robotic user interfaces (MRRUIs). By analyzing existing literature and conducting a card sorting experiment, the authors provide a structured approach to understand the complex interplay of factors in MRRUI design, aiming to improve clarity, consistency, and effectiveness in this rapidly growing research area.

What Problem Does This Paper Address?

The field of mixed reality robotic user interfaces (MRRUIs) is experiencing exponential growth, with an increasing number of publications combining robotics and mixed reality. However, the interdisciplinary nature of MRRUIs makes them complex to design and analyze. While individual aspects have been theoretically structured, there's a lack of a comprehensive taxonomy that integrates all relevant factors. This absence hinders a clear understanding, detailed description, and easy comparison of newly developed MRRUIs, making it challenging for researchers and developers to navigate this complex landscape.

Methods

The authors developed the IMPAct framework through a multi-step process:

  1. Literature Investigation: They conducted an extensive review of high-impact publications in fields like 3D interaction, mixed reality, user interfaces, and robot autonomy. From these, they extracted 62 technical factors that directly influence MRRUI design.
  2. Card Sorting Experiment: To cluster these factors into meaningful groups, they organized an open card sorting experiment with 9 experts from HCI research. Participants were asked to group the factors, leading to the identification of 13 initial groups.
  3. Hierarchical Clustering: The authors then further clustered these 13 groups into four major categories: Interaction, Mediation, Perception, and Acting. This hierarchical structure aims to provide a memorable and practical framework.
  4. Framework Development: The resulting IMPAct framework categorizes factors that are objective, technical, and controllable by system designers, thus avoiding reliance on subjective user mental models.

Datasets

Not discussed in the paper. The study relies on a literature review and a card sorting experiment with human experts, not on specific datasets for training or evaluation.

Key Insights and Ideas

  • The Need for a Unified Framework: The rapid growth of MRRUI research highlights the urgent need for a standardized way to classify and design these complex systems.
  • Objective, Technical Factors are Key: Focusing on measurable, technical factors controllable by designers, rather than subjective user perceptions, makes the framework more objective and universally applicable.
  • Holistic Approach is Crucial: MRRUI design is not just about the interface; it involves a complex interplay of how users interact, how information is mediated, how users perceive the environment, and how the robot acts. The IMPAct framework captures this holistic view.
  • Memorable Structure Enhances Usability: The four-category structure (Interaction, Mediation, Perception, Acting) is designed to be memorable, facilitating easier adoption and use by researchers and designers.
  • Framework as a Design Tool: Beyond classification, IMPAct serves as a valuable tool for designing new MRRUI systems by ensuring all relevant aspects are considered.

Key Results

The paper presents the IMPAct framework, which consists of four main categories:

  • Interaction: Covers how the user and robot communicate and control each other, including parameters like action type, control alignment, and interaction reality.
  • Mediation: Focuses on how information is presented and received, encompassing aspects like display type, input methods, and output mechanisms.
  • Perception: Deals with how the user experiences the mixed reality environment, including embodiment, immersion, modalities, and spatial/temporal aspects.
  • Acting: Describes the robot's capabilities and behavior, such as autonomy, its appearance, and its intended application.

The authors validate the framework by applying it to classify a specific pick-and-place MRRUI system, demonstrating its ability to provide detailed and precise descriptions. The validation section provides a comprehensive breakdown of the analyzed system across all IMPAct categories.

Limitations and Future Work

  • Potential for Missing Factors: Despite extensive literature review, the authors acknowledge that some relevant aspects might have been missed, and the framework may need adaptation as the field evolves.
  • Efficiency of Assessment: While the framework enables detailed descriptions, assessing the values of all factors can be time-consuming. Future work could involve developing a wizard-based GUI tool with explanations and links to further resources to streamline this process.
  • Qualitative Aspects: The current framework focuses on objective, technical measures. Future extensions could explore how these objective factors relate to qualitative aspects (e.g., user experience, satisfaction) and how to integrate them into a holistic re-design process.
  • MRRUI Archive: The authors suggest that the IMPAct framework could serve as a foundation for creating a dedicated archive of MRRUI systems, similar to existing robotic web archives.

Summary updated

BibTeX
@article{krupke2019impact,
  author = {Dennis Krupke and Jianwei Zhang and Frank Steinicke},
  title = {{IMPAct: A Holistic Framework for Mixed Reality Robotic User Interface Classification and Design}},
  journal = {Multimodal Technol. Interact.},
  year = {2019},
  doi = {10.3390/mti3020025},
}

Loading related content...

Figure