Abstract
Autonomous systems, from self-driving cars to medical robots, are traditionally designed to handle a predefined set of scenarios. However, the real world is inherently unpredictable, presenting "unknown unknowns" that designers cannot foresee. This paper proposes Autonomics, a new engineering foundation that shifts the focus from "predicting the rare" to "engineering for the unpredicted" through principles like self-reflection, social mimicking, and proactive probing.
The Challenge of the "Unknown Unknown"
Engineers often strive for exhaustive scenario coverage, but predicting every possible condition is a fool's errand. Environmental complexity, post-deployment changes, and the confluence of rare events create novel situations that lead to catastrophic failures in systems restricted by rigid programming. The core problem is designing systems to handle situations that the designers themselves have not imagined.
The Autonomics Framework
The authors outline a conceptual framework for "Autonomics" based on three primary pillars:
1. Reactive and Proactive Behaviors
Systems should be governed by high-level, abstract rules that prioritize safety when uncertainty is high (e.g., "if the situation is ambiguous, slow down"). Furthermore, systems should engage in "probing"—actively interacting with unknown objects to gather data, such as a robot arm testing the weight of an unidentified container.
2. Knowledge and Skill Acquisition
Rather than relying solely on pre-programmed logic, autonomous systems need "general world knowledge" (such as a physics engine) and the capacity to acquire new information at run-time from sensors or external knowledge bases.
3. The System as a Social Entity
Autonomous systems do not operate in a vacuum. By treating them as social entities, designers can implement behaviors like "asking for help" from humans or "mimicking" the actions of others in the environment.
Social Mimicking and the Tsunami Lesson
A pivotal example of social intelligence in the face of the unexpected is the behavior of motorists during the 2011 Japanese tsunami. Many drivers escaped not because they saw the wave themselves, but because they observed other drivers turning around and followed suit.
This "social common sense" is a powerful tool for autonomous systems. If a self-driving car observes all surrounding vehicles performing an unusual maneuver, such as a collective U-turn, it should be designed to mimic that behavior even if its own sensors haven't yet identified the underlying cause.
Theoretical Foundations: Ontological Deficit
The paper provides a formal definition of the "unexpected" as an ontological deficit. This occurs when a system's internal model (its ontology) either lacks a representation for a real-world object or fails to map a perceived object to an internal class at run-time. To address this, systems must employ self-reflection, analyzing their own history to recognize states that previously led to failure and aborting risky actions.
Defining the Responsibility Space
Handling the unexpected is as much a policy challenge as a technical one. Designers must pre-define the "responsibility space"—the scope of what the system is expected to handle versus what requires external intervention. For instance, if an autonomous taxi encounters an unpassable road, is it responsible for rerouting or for notifying a central dispatcher to manage the fleet?
Conclusion and Future Directions
The "Science of the Unexpected" is in its early stages. While the proposed principles of Autonomics provide a roadmap, significant work remains to bridge the gap between high-level theory and verifiable software. Future research will focus on creating domain-specific ontologies, improving simulation tools to include "messy" real-world scenarios like street performers or chaotic crowds, and developing rigorous specification libraries for resilient autonomous behavior.
Summary updated