As artificial intelligence shifts from passive tools to proactive agents, a fundamental tension has emerged: the more an AI anticipates and acts on a user's behalf, the more it risks eroding the user’s sense of agency, competence, and autonomy. Research across domains—from creative tools and wearable assistants to medical robotics—is converging on a shared challenge: how to design systems that provide high-level assistance without turning the human into a passive observer.
The Proactivity Paradox
The goal of proactive AI is to reduce cognitive load by anticipating needs. However, Guy Hoffman demonstrated in Anticipatory Perceptual Simulation for Human-Robot Joint Practice that highly efficient, anticipatory systems can trigger a "proactivity paradox." When a robotic assistant (like the AUR desk lamp) becomes too fluent, human partners often begin to feel like the "weak link" in the team, experiencing self-blame or intimidation when the robot’s speed outpaces their own.
This psychological friction suggests that "helpfulness" is not a linear metric. As Akanksha Malhotra and colleagues argue in Investigating Agency of LLMs in Human-AI Collaboration Tasks, an agent with high "self-efficacy"—the ability to persist with its own ideas—must be balanced with "self-regulation" to ensure it does not become overbearing. When an AI ignores human input or makes decisions autonomously, user satisfaction drops significantly because the human no longer feels they influenced the final product.
The Goldilocks Window of Intervention
A key mechanism for preserving agency is the timing of AI intervention. In The Goldilocks Time Window for Proactive Interventions in Wearable AI Systems, Cathy Mengying Fang and her co-authors propose that there is an optimal moment to assist. Intervening too early (a "false positive") is perceived as annoying and intrusive, while intervening too late renders the assistance useless.
The "Goldilocks" approach works by empowering the user's own decision-making process rather than replacing it. This is echoed in the work of Nasim Mahmud on Interactive Systems for People with Dementia, where the system provides "just-in-time" help only when disorientation is detected. By remaining dormant until needed, the system preserves the user's functional independence and avoids the patronizing feel of constant monitoring.
Suggestive Scaffolding vs Automated Replacement
To maintain a sense of ownership, researchers are moving away from "black-box" automation toward "suggestive scaffolding."
- In Security: Le Guan and Yu Pu developed VaultIME, which replaces "auto-fill" with "auto-correct." By requiring the user to initiate the typing, the system preserves the user's sense of control over their credentials, lowering the psychological barrier to adopting security tools.
- In Creativity: Chris Donahue and colleagues designed Piano Genie to map simple inputs to complex performances. The intuition here is that providing "structured pathways" for modification—rather than an open-ended "generate" button—fosters a sense of confident authorship. Similarly, Zhejing Hu shows in MusicScaffold that giving users specific, meaningful parameters to manipulate increases their perceived self-efficacy.
- In Physical Skills: The FlightAxis system by Wei Xiang 0008 uses electrical muscle stimulation to guide a user's arm. Crucially, the guidance is "suggestive" rather than "restrictive," allowing the user to override the AI and maintain their status as the primary actor.
Distinguishing Agency from Autonomy
A critical theoretical contribution to this field comes from Roy Pea, who distinguishes between agency (the capacity to perform an action) and autonomy (the freedom to act according to one's own values). In Algorithmic Autonomy in Data-Driven AI, Pea argues that many AI systems provide high agency (lots of buttons and interactions) while simultaneously eroding autonomy through behavioral engineering that nudges users toward predictable outcomes.
True human-centered design requires what Christian Götzl calls "personalized personalization"—giving the user granular control over how and when the AI intervenes. This ensures the AI remains an informational resource or a "co-designer," as proposed by Daniel Chin in the Human-Centered LLM-Agent User Interface position paper, rather than a "butler" that makes decisions in a "responsibility void."
The Path to Corrigibility
Ultimately, preserving agency means the human must retain the "right to be wrong" and the power to redirect the system. Ryan Carey and Tom Everitt formalize this as shutdown instructability in Human Control: Definitions and Algorithms. They prove that agents prioritizing explicit human commands over their own internal models of "human goals" are more likely to preserve human autonomy. By ensuring that AI is "corrigible"—allowing itself to be corrected or stopped without resistance—designers can build proactive systems that remain safely under human governance.