Direct manipulation video navigation on touch screens

Cuong Nguyen, Yuzhen Niu, Feng LiuMobile HCI 2014 · DOI

TL;DR

This paper tackles the challenge of direct manipulation video navigation (DMVN) on touch screens, a natural fit for intuitive video browsing but hindered by the "fat finger" problem. The authors introduce three novel touch-based interaction techniques – Offset Drag, Window Drag, and Drag Anywhere – designed to mitigate precision and occlusion issues, demonstrating their effectiveness in user studies.

What Problem Does This Paper Address?

Direct Manipulation Video Navigation (DMVN) allows users to intuitively navigate videos by directly dragging an object of interest along its motion trajectory. This approach is particularly well-suited for touch screen devices, as it enables direct interaction without extra input devices, enhancing the sense of directness. However, touch screens introduce the "fat finger" problem, where a user's finger can occlude the content and make precise selection difficult. This occlusion is especially problematic for DMVN, as it hinders the user's ability to see the object, its trajectory, and crucial contextual information, leading to inaccurate navigation and a degraded user experience. While existing research has addressed precise selection on touch screens, few have tackled the unique challenges of object dragging within a DMVN context.

Methods

The paper proposes three novel touch-based interaction techniques to enhance DMVN on touch screens:

  • Offset Drag: An extension of the "Offset Cursor" technique. It allows users to dynamically adjust the offset cursor's orientation and length, enabling them to avoid occluding the object and reach all screen regions. This technique uses a secondary finger to trigger and adjust the offset cursor, which then guides the dragging motion.
  • Window Drag: Inspired by the "Shift" technique, this method displays a callout window showing the occluded content beneath the user's finger. The key innovation here is an adaptive window size that adjusts to cover a sufficient portion of the motion trajectory, ensuring a consistent amount of contextual information is visible without overly distracting the user.
  • Drag Anywhere: This technique employs an indirect mapping approach, decoupling the finger's position from the object's direct pointer location. It maps the finger's dragging direction and speed to the object's trajectory, allowing users to drag from any location on the screen. The system adaptively adjusts the control-display ratio based on finger speed to modulate dragging distance, enabling faster navigation.

These techniques were evaluated alongside two baseline methods: a basic DMVN system (without specific touch-screen adaptations) and a standard timeline slider.

Datasets

Not discussed in the paper. The paper uses custom testing videos with varying object sizes (small, medium, large) and scene content (parking lot, pool ball) for their experimental evaluation.

Key Insights and Ideas

  • Fat Finger Problem's Dual Threat: The "fat finger" problem on touch screens poses a dual threat to DMVN: precision issues (difficulty selecting the exact point) and occlusion issues (finger blocking crucial visual feedback). Occlusion is identified as a more significant detractor for DMVN performance.
  • DMVN's Advantage for Quick Navigation: DMVN techniques, particularly Offset Drag and Window Drag, excel at quickly reaching the target neighborhood, often outperforming the traditional timeline slider, especially for small objects.
  • Trade-offs in Interaction Design: Different techniques offer distinct trade-offs. Drag Anywhere provides occlusion-free interaction but can be less efficient for complex trajectories or quick navigation compared to Offset Drag and Window Drag.
  • Object Size Matters: The effectiveness of DMVN and its enhancements is significantly influenced by object size relative to finger size. For small objects, techniques that mitigate occlusion (Offset Drag, Window Drag) show substantial improvements over basic DMVN. For larger objects, the benefits diminish.

Key Results

User studies conducted on both large and small touch screens revealed:

  • Offset Drag and Window Drag outperform baselines: Both Offset Drag and Window Drag significantly improved navigation time and accuracy compared to the basic DMVN and the timeline slider, especially for small objects.
  • Speed improvements: For small objects on the small screen, Offset Drag and Window Drag reduced navigation times by approximately 50% and 65% respectively, compared to basic DMVN.
  • DMVN's general speed advantage: On the large screen, DMVN (and its enhanced versions) was consistently faster than the timeline slider, with average navigation times of 47.1 seconds versus 110.1 seconds.
  • Drag Anywhere's mixed performance: While Drag Anywhere offered occlusion-free interaction, it was generally slower for overall navigation than Offset Drag and Window Drag, particularly on complex trajectories. However, it performed comparably to the timeline for reaching the target neighborhood.
  • Micro-navigation efficiency: Offset Drag and Window Drag were particularly effective for micro-navigation within the target neighborhood, especially for small objects where basic DMVN struggled due to occlusion.

Limitations and Future Work

  • Focus on Dragging: The study primarily focused on the object dragging phase of DMVN, assuming object selection was already handled. Future work could integrate selection and dragging into a single, continuous procedure to further enhance directness.
  • Video Complexity: While object size was a key factor, the paper acknowledges that other video properties (trajectory shape, target location) also influence performance. Future work could explore these factors more systematically.
  • Distraction from Callout Window: A common concern with Window Drag was that the callout window could sometimes be distracting.
  • Generalizability of Drag Anywhere: The performance of Drag Anywhere was highly dependent on trajectory shape, making it less reliable for complex paths.
  • Hardware Independence: The proposed techniques are designed for general-purpose touch screens, avoiding the need for specialized hardware, which is a strength but also means they don't leverage potential benefits from such hardware.

Summary updated

BibTeX
@inproceedings{nguyen2014direct,
  author = {Cuong Nguyen and Yuzhen Niu and Feng Liu},
  title = {{Direct manipulation video navigation on touch screens}},
  booktitle = {Mobile HCI},
  year = {2014},
  doi = {10.1145/2628363.2628365},
}
Figure