This paper, recently presented by researchers from Imperial College London, King’s College London, and York University, explores how multimodal haptic feedback—specifically vibration and variable softness—affects the cognitive state and confidence of operators teleoperating Remotely Operated Vehicles (ROVs) in challenging underwater environments.
TL;DR
While haptic feedback for ROV teleoperation increases mental demand and frustration by making operators more aware of invisible hazards, it significantly boosts user confidence and mitigates the physical and temporal stress caused by environmental cross-currents. Operators receiving haptic cues drove significantly faster and more accurately than those relying on visual information alone.
What Problem Does This Paper Address?
Teleoperating an ROV is a high-stakes task. Unlike ground robots, ROVs operate in a 3D space where hazards like cross-currents are often invisible to onboard cameras. Furthermore, underwater visibility is frequently compromised by turbidity and poor lighting.
While previous studies have shown that haptic feedback can improve "hard" performance metrics like positioning accuracy, we know very little about the "soft" psychological effects on the operator. Does adding a haptic channel overwhelm the pilot? Does it actually make them feel more confident? This paper addresses the gap between quantitative task performance and the nuanced cognitive load experienced by the human in the loop.
Methods
The researchers conducted an in-water study using a BlueROV2-Heavy in a controlled pool environment. Participants were tasked with driving the ROV in a straight 4-meter path toward a target buoy.
The core of the experiment was a custom planar haptic touchpad that provided two distinct types of feedback:
- Softness Change: Achieved via particle jamming (where a vacuum is used to change the stiffness of a granular material), this indicated the distance to the target buoy.
- Asymmetric Vibration: This indicated the presence and direction of a cross-current generated by a fixed thruster in the pool.
Operators used the same touchpad for input, where finger position controlled steering and the magnitude of downward force controlled speed. Cognitive load was measured using the NASA Task Load Index (TLX), and average ground speed was used as a proxy for operator confidence.
Datasets
The study involved 8 novice participants (5 male, 3 female) aged 20–70. The primary data collected consisted of:
- NASA TLX Scores: Self-reported across six dimensions (mental, physical, temporal, effort, performance, and frustration).
- ROV Telemetry: Ground speed logs and positioning data across three conditions: Control (still water), Cross-current (no haptics), and Haptics + current.
Key Insights and Ideas
- The Frustration Paradox: Surprisingly, the haptic condition was judged to be the most frustrating and mentally demanding. The authors suggest this is because haptics make the operator acutely aware of the hazard. In the "current-only" condition, operators might be struggling without fully realizing why, leading to lower reported frustration but worse performance.
- Knowledge is Confidence: Despite the higher mental workload, operators drove the ROV significantly faster when receiving haptic cues. This suggests that "knowing" the environment—even if that knowledge is taxing to process—builds the confidence necessary for decisive navigation.
- Stress Mitigation: The introduction of a cross-current naturally spikes physical and temporal demand. Haptic feedback acts as a buffer, bringing these specific stressors down toward control levels, even if it shifts that load into the "mental demand" category.
Key Results
- Confidence Boost: Participants drove significantly faster with haptic feedback (0.388 m/s) compared to the current-only condition (0.291 m/s) or even the control condition (0.362 m/s).
- Cognitive Load Composition: While the total NASA TLX score was only slightly lower with haptics (31.31) than without (32.07), the type of load changed. Haptics improved self-assessed performance and reduced physical/temporal demand compared to the current-only condition.
- Learning Effects: As seen in the speed data, the benefits of haptics were most pronounced in the "Combined" results, indicating that as users became more experienced, they leveraged the haptic information more effectively.
Limitations and Future Work
- Novice vs. Expert: The study used novice operators. Experienced ROV pilots might have a higher baseline for cognitive load, potentially allowing them to reap the benefits of haptics without the initial "frustration spike."
- Predictability of Hazards: The cross-current was generated by a fixed thruster. Real-world underwater hazards are dynamic, 3D, and unpredictable.
- Post-hoc Measurement: The NASA TLX is a retrospective questionnaire. The authors suggest that future work should use real-time monitoring of cognitive load (e.g., physiological sensors) to better correlate load spikes with specific environmental events.
- Hardware Latency: Future iterations could benefit from pump-free particle jamming to provide faster response times and better integration with other feedback modalities.
Summary updated