TL;DR
Standard Mixed Reality (MR) instructions often focus on the "what" (the current step) but ignore the "why" (how it affects the future). This study introduces a three-level hierarchy for visualizing intention-driven causality—event, interaction, and gesture—and finds that while showing these causal links increases initial learning time, it significantly improves task performance and user comprehension during execution.
Introduction
When learning manual tasks like assembly or repair in Mixed Reality (MR), most systems provide "rote" guidance—step-by-step instructions that learners follow without necessarily understanding the underlying logic. While efficient for immediate completion, this approach often fails to foster true skill acquisition. Learners may finish a task without grasping how a current action, such as tightening a specific bolt, impacts a later stage, like ensuring structural stability for a mounted component.
Human learning is naturally tied to causality. The authors address this gap by proposing a systematic framework for representing and visualizing cause-and-effect relationships in MR, moving beyond simple guidance toward effective skill learning.
The Intention-driven Causality Framework
The researchers developed a framework based on Intention-driven Causality, which decomposes tasks into a three-level hierarchy:
- Event Level: High-level goals (e.g., "Assemble the base").
- Interaction Level: Specific actions within an event (e.g., "Aligning the bolts").
- Gesture Level: Fine-grained hand-object affordances (e.g., "Grasping the bolt at a specific angle").
By visualizing these levels, the system communicates the expert's intention to the learner, showing not just the "Cause" (current step) but also the "Effect" (the future step it enables).
System Implementation and Task Design
The system utilized an Oculus Quest 2 with a ZED Mini stereo camera for passthrough MR. A custom "Test Bed" was created to display a visualization graph of causal links. To evaluate the framework, the authors designed a complex Camera Assembly Task involving 5 major events and 24 interactions, requiring tools such as drills, hammers, and 3D-printed components.
Experimental Results
A user study with 48 participants compared four groups: a control group (GC) shown only the current step, and three groups (G1, G2, G3) shown increasing levels of causal information.
The Learning/Performance Trade-off
There is a clear "tax" on initial learning. Participants in the most informed group (G3) took longer to finish the learning phase (22.27 min) compared to the control group (18.24 min). However, this investment paid off during the memory-based testing phase:
- Testing Speed: G3 completed the task in 4.31 minutes, significantly faster than GC's 5.20 minutes.
- Instruction Dependency: Despite the increased information density, G3 participants revisited tutorials less frequently (2.33 times) than the control group (3.17 times).
- Comprehension: G3 reported a significantly higher understanding of why actions were necessary (scoring ~4.4/5 vs ~2.6/5).
Discussion and Future Work
The study suggests that visualizing high-level causality alone (G1) is insufficient; without the "deeper" details of interactions and gestures, causal links are harder to internalize. By providing a multi-level view, MR systems can help users build more robust mental models.
Future work includes exploring Dynamic Modeling to show the adverse effects of mistakes (e.g., "If you don't tighten this now, the arm will fall off later") and developing AI-driven adaptive visualizations to manage cognitive load by showing causal links only when necessary.
Summary updated