Abstract
Highly automated robot ecologies (HARE), or societies of independent autonomous robots or agents, are rapidly becoming an important part of much of the world's critical infrastructure. As with human societies, regulation, wherein a governing body designs rules and processes for the society, plays an important role in ensuring that HARE meet societal objectives. However, to date, a careful study of interactions between a regulator and HARE is lacking. In this paper, we report on three user studies which give insights into how to design systems that allow people, acting as the regulatory authority, to effectively interact with HARE. As in the study of political systems in which governments regulate human societies, our studies analyze how interactions between HARE and regulators are impacted by regulatory power and individual (robot or agent) autonomy. Our results show that regulator power, decision support, and adaptive autonomy can each diminish the social welfare of HARE, and hint at how these seemingly desirable mechanisms can be designed so that they become part of successful HARE.
Introduction
As we transition toward a world dominated by driverless cars, smart grids, and automated high-frequency trading, we are effectively creating Highly Automated Robot Ecologies (HARE). These are collections of independent, autonomous agents owned by different stakeholders who must share constrained resources. Unlike traditional supervisory control, where a human can directly override a robot, a regulator in a HARE influences behavior indirectly—for example, by adjusting tolls or pricing. This study investigates how regulatory power and robot adaptivity affect a human's ability to achieve societal goals, such as maximizing traffic flow or ensuring fair resource distribution.
Methodology
The researchers conducted three distinct human-in-the-loop simulation studies:
- Study 1 (Transportation): Participants regulated 300 simulated cars in a network by setting tolls to maximize throughput.
- Study 2 (Resource Management): Participants regulated 8 "smart" apartment tenants sharing a limited water supply by setting per-unit water prices.
- Study 3 (Decision Support): A follow-up to Study 1 that introduced a warning system to forecast congestion.
Studies 1 and 2 utilized a 2x3 between-subjects design, varying Robot Adaptivity (Simple automation with fixed rules vs. Adaptive automation using reinforcement learning) and Regulatory Power (None, Limited, or Unlimited).
The Paradox of Regulatory Power
A primary finding of the research is the "Power Paradox." In scenarios involving simple automation, regulators with limited power consistently outperformed those with unlimited power. When granted unlimited authority, human regulators tended to over-intervene, making frequent adjustments without allowing the system to stabilize. This behavior prevented them from forming accurate mental models of the underlying agent behavior.
As shown in the data, societies of simple robots with limited regulatory power achieved nearly 80% of optimal throughput, while those with unlimited power dropped to approximately 60%. Interestingly, adaptive robots performed better than simple robots when there was no regulation, but they proved much harder for humans to regulate effectively. Because adaptive robots constantly changed their behavior in response to the environment, the human regulator was essentially chasing a moving target.
Mental Models and Over-Intervention
The failure of unlimited power is linked to the frequency of intervention. Regulators with unlimited power made significantly more toll adjustments per second. This high frequency of change correlated with a failure to understand the system's natural tendencies. Post-experiment questionnaires revealed that these participants were significantly less likely to correctly identify which nodes or time slots the robots preferred.
Furthermore, Study 3 demonstrated that automated decision support can be counterproductive. Providing a forecasting warning system actually decreased performance for simple robots. Regulators began relying on the warnings as a crutch, reacting to immediate symptoms rather than learning the system's long-term dynamics.
Conclusion
These studies highlight the concept of "neglect benevolence," where refraining from intervention allows a regulator the necessary time to observe and model a system's natural state. Future HARE regulation systems should be designed not just to provide warnings, but to actively assist humans in building robust mental models of autonomous agent behavior, balancing the need for intervention with the necessity of observation.
Summary updated