TL;DR
This paper explores a novel "eyes-free" mobile interaction system for geosocial networking, allowing users to track and interact with remote agents using only non-visual feedback. An experiment demonstrated that users could successfully track simulated agents, highlighting the potential of this approach for future mobile social applications.
What Problem Does This Paper Address?
The proliferation of mobile phones with built-in sensors like GPS, accelerometers, and magnetometers opens up exciting possibilities for location-based and social interactions. However, current systems often rely on visual displays, which can be impractical or undesirable in many mobile scenarios. This paper addresses the challenge of enabling intuitive, eyes-free interaction with remote agents in a shared physical and virtual space, specifically focusing on how users can track and engage with others without direct visual cues. The authors aim to explore the feasibility of such systems and understand the design considerations, particularly regarding how to represent and handle the inherent uncertainties in sensor data and user intent.
Methods
The core of the system is a "virtual probe" controlled by the user's mobile device. The device's orientation (pitch and roll) is used to estimate the direction and distance of the virtual probe. Pitching the device forward extends the probe into the virtual environment, while pitching it back retracts it. The system uses GPS for location and a combination of magnetometers and accelerometers to determine bearing and compensate for tilt. Feedback is provided through audio and haptic cues when the virtual probe "collides" with objects or agents in the virtual environment.
A key methodological innovation is the explicit representation and handling of uncertainty. Instead of presenting a single, precise estimate of the user's or agent's position, the system uses a probabilistic approach (Monte Carlo sampling) to display a cloud of possible future positions. This acknowledges the inherent inaccuracies in sensor data and user behavior, aiming to provide a more realistic and manageable interaction experience.
The experiment involved participants tracking a simulated agent exhibiting three distinct behaviors: random walk, attention check (short jumps), and goal-directed movement. Participants were blindfolded and used the eyes-free interface to locate and follow the agent.
Datasets
Not discussed in the paper. The paper describes an experiment using a simulated agent and a controlled virtual environment.
Key Insights and Ideas
- Eyes-free interaction is feasible for tracking: The experiment demonstrated that users, even with minimal experience, could track a moving agent using only audio and haptic feedback, suggesting that eyes-free interaction is a viable paradigm for mobile social applications.
- Uncertainty representation is crucial: Explicitly displaying uncertainty in position and intent is not just a technical necessity but a design feature that can improve user experience and enable more robust interaction. It allows users to understand the limitations of the system and adapt their strategies.
- Metaphor matters for feedback: The pitch-based metaphor for distance scanning proved challenging for users. While heading tracking (left-right movement) was intuitive and well-supported by audio panning, the forward-backward movement (distance) was harder to perceive and control, highlighting the sensitivity of interaction design to the choice of metaphors and feedback mechanisms.
- Social cognition in geosocial interaction: The paper touches upon the potential for social cognition (e.g., joint attention, imitation) to emerge in these systems, suggesting that understanding how humans naturally infer intentions and share focus can inform the design of richer and more intuitive geosocial interactions.
Key Results
- Tracking success: 9 out of 13 participants successfully completed the entire tracking task with all three goals. 4 participants achieved at least one goal in the goal-directed phase.
- Heading vs. Distance Tracking: Participants generally performed better when tracking the agent's heading (left-right movement) than its distance (forward-backward movement). This is attributed to more effective feedback for heading (audio panning) compared to distance.
- Performance variation: A wide range of performance was observed, with some participants showing very precise tracking and others covering a much larger area, indicating significant differences in user ability and strategy.
- Error analysis: Figures 11(a) and 11(b) show the normalized mean squared error for heading and tilt tracking, respectively. Participants with lower mean error and standard deviation performed better.
Limitations and Future Work
- Simulated environment: The experiment was conducted in a controlled, indoor environment with a simulated agent. Future work should involve real-world testing with actual human users to validate the findings and explore more complex social dynamics.
- Single-user interaction: The experiment focused on a single user tracking a simulated agent. Expanding to multi-user interaction is a key next step to observe how people engage and disengage in remote contact.
- Distance feedback: The current pitch-based metaphor for distance scanning needs improvement. Future work could explore alternative metaphors or more explicit feedback mechanisms for forward-backward movement.
- Generalization to hand movements: The current system tracks device orientation. Generalizing this to more expressive hand movements and their social implications requires further investigation.
- Subtle social cues: The paper suggests that subtle movements of a mobile phone could convey significant information about mood and intentions. Further research is needed to explore how these subtle cues can be leveraged in geosocial interaction.
Summary updated