When we design assistive robots, smart user interfaces, or autonomous vehicles, our primary objective is usually optimization: we want to minimize task completion time, reduce physical or cognitive workload, and eliminate human error. However, a recurring friction emerges across these domains. As systems become more proactive and efficient, they frequently erode the user's sense of agency—the subjective feeling of being the author of one's own actions and choices.
This tension is not merely a psychological footnote; it is a critical failure mode for system design. When an autonomous system overrides direct human input or acts unpredictably, users experience frustration, anxiety, and a loss of trust, often leading them to abandon the technology entirely (Mohammed et al.; Falqueto et al.). To build effective human-machine partnerships, we must understand why this tradeoff occurs and how we can design systems that maximize performance without compromising human autonomy.
Why Efficiency Threatens Agency
To understand the mechanics of this tradeoff, we must look at how automation alters the control loop. In a manual system, there is a direct, predictable mapping between user intent, motor input, and system output. In an automated or shared-control system, the machine introduces its own predictive models and autonomous execution layers.
This decoupling manifests in several distinct ways:
- Overriding Direct Commands: In advanced prosthetics, autonomous features can stabilize a grasp or modulate muscle activity via reflex loops, but they often do so by overriding the user's immediate muscular signals, stripping away the user's sense of embodiment (Sagastegui Alva et al.).
- Unpredictable Proactivity: When a system anticipates a user's needs and acts ahead of time—such as a GUI agent orchestrating multi-app workflows (Shi et al.) or a robot arm moving toward a predicted target (Shared Autonomy via Hindsight Optimization)—any misalignment between the system's prediction and the user's actual intent destroys the feeling of control.
- Authority Conflicts: In teleoperation or shared navigation, a "traded control" model can make the user feel like they are actively competing against the machine rather than collaborating with it, leading to cognitive friction and confusion (Oh et al.).
Paradoxically, making a system too fluent or fast can backfire. Hoffman demonstrated that highly responsive, anticipatory robots can intimidate human partners, causing them to experience self-blame when their own pace fails to match the machine's speed. Similarly, decoupling physical movement from user input via autonomous path-following can induce severe user anxiety regarding potential collisions, even if the system is mathematically safe (Plümer et al.).
Mechanisms for Preserving Agency
How do we break this zero-sum dynamic? Research points to several architectural patterns that preserve human agency while retaining the benefits of automation.
1. Error-Gated Intervention and Shared Authority
Instead of constantly blending or overriding human inputs, systems can adopt a "guardrail" approach. Wen et al. argue that driving automation maintains a strong sense of agency when it intervenes only to correct critical errors, allowing successful human intentions to execute unhindered.
This insight is mirrored in physical assistance: Falqueto et al. developed a robotic walker framework that classifies whether a user's movement is natural and steps in only when the trajectory becomes erratic or unsafe. By leaving the successful execution of intent to the user, the system enhances safety without eroding the subjective feeling of control.
2. Proactive Pausing and Tiered Autonomy
Autonomy should not be a binary toggle. Instead, intelligent agents must know when to pause and yield control. Jennings proposed a tiered feedback loop where an agent operates independently during routine tasks but explicitly halts to request user intervention at critical decision points, preserving human veto power.
This mechanism is highly effective in modern user interfaces. For instance, Peng et al. introduced a UI agent that proactively pauses when encountering ambiguous choices (e.g., multiple products matching a description) rather than making silent, arbitrary decisions on behalf of the user. This balances operational efficiency with the precision of human preference (Kahlon et al.).
3. Goal-Congruent Feedback and Empowerment Maximization
A user does not necessarily need to control every low-level motor action to feel in control. Wen et al. found that a user's sense of agency is preserved as long as the system's actions remain strictly congruent with their high-level goals.
Taking this a step further, Du et al. proposed a paradigm shift: instead of trying to predict and execute a specific human goal (which risks frustrating the user if the prediction is wrong), an assistive system should maximize the human's empowerment—their capacity to influence the environment and reach any potential future state. By keeping the user's options open rather than forcing a pre-calculated path, tasks feel significantly easier while the user retains full strategic autonomy.
Traditional Shared Autonomy:
[User Input] ───► [Goal Prediction] ───► [Autonomous Execution] ───► (Potential Misalignment / Agency Loss)
Assistance via Empowerment:
[User Input] ───► [Empowerment Engine] ───► [Expand Future Options] ───► (User Retains Choice / Agency Preserved)
Designing for Empowerment Over Pure Utility
Ultimately, the goal of assistive and automated systems should be functional empowerment rather than total delegation. Whether designing social robots for older adults (Ostrowski et al.), productivity tools for neurodivergent students (Salam), or digital mental health interventions (Health-Promoting Effects and Everyday Experiences With a Mental Health App Using Ecological Momentary Assessments and AI-Based Ecological Momentary Interventions Among Young People: Qualitative Interview and Focus Group Study), the technology must serve as a tool that enhances human capability, not a restrictive constraint that replaces human decision-making (Micoulaud-Franchi et al.).
By measuring the user's internal perception of control—potentially even through neural signatures like alpha-mu rhythm suppression (Measurement of the Perception of Control during Continuous Movement using Electroencephalography)—future systems can dynamically modulate their proactivity, ensuring that as efficiency scales upward, human agency is left intact.