Introduction
In high-stakes environments like aviation and spaceflight, spatial disorientation remains a critical threat to safety. While artificial intelligence (AI) can offer precise corrections, a significant gap often exists between AI-driven strategies and human intuition. This research explores "dyadic" human-in-the-loop (HITL) learning as a method to develop AI models that not only solve control tasks but do so using strategies aligned with human sensorimotor constraints.
The Challenge of AI Embodiment
Standard reinforcement learning (RL) models often achieve "superhuman" performance by exploiting capabilities humans lack, such as near-zero reaction times or extreme control stiffness. However, if an AI's guidance feels "alien," a disoriented pilot is less likely to trust it during a crisis. The goal is to achieve "embodiment"—creating an AI that understands the specific challenges and limitations a human operator faces.
As shown in Figure 1, deep learning (DL) models trained on human data are far better at predicting actual human joystick deflections than pure RL models. This suggests that DL models can better "embody" human strategies, making them more suitable for collaborative tasks.
Methodology: Dyadic Learning and the VIP Task
The researchers utilized a Virtual Inverted Pendulum (VIP) task where participants used a joystick to balance a simulated system upright. To simulate disorientation, the visual display was "50% coherent," meaning half the visual cues moved randomly, making motion perception difficult.
The study followed a three-stage "dyadic" approach:
- Pre-training: AI models were trained on the MARS dataset, containing physical balance data from 34 subjects across various skill levels.
- Dyadic HITL Training: A human and AI were paired. The AI attempted the task while the human could override its inputs. These disagreements were logged to fine-tune the AI, teaching it to align with human control patterns.
- AI Guidance: The fine-tuned AI then provided real-time visual guidance (red arrows) to assist human operators.
Key Results and Behavioral Convergence
The experiments revealed that dyadic training significantly improved the AI's ability to assist humans. Notably, models trained on a broad range of human proficiencies ("All Proficiency") showed the greatest gains after HITL fine-tuning compared to those trained only on expert data.
As illustrated in Figure 6, the fine-tuned assistants helped humans reduce "destabilizing actions"—moves that worsen a potential crash—and lowered overall oscillation. Phase-plot analysis further confirmed "behavioral convergence," where the AI and human began to react in similar patterns, effectively speaking the same "sensorimotor language."
Conclusion and Future Directions
This proof-of-concept demonstrates that human-in-the-loop training can bridge the embodiment gap, creating AI partners that are more intuitive and effective. Future work aims to expand this to multi-axis flight simulators and explore multimodal guidance, such as haptic feedback or natural language explanations, to further deepen the human-AI partnership in critical control tasks.
Summary updated