Abstract
Traditional approaches to artificial intelligence often treat embodiment and mortality as obstacles to be overcome or ignored. This paper argues the opposite: that the constraints of physical reality—specifically vulnerability and mortality—are essential for developing agents capable of robust generalization and genuine care. By formalizing existential concepts such as "being-towards-death" as homeostatic drives within a non-ergodic reinforcement learning framework, we demonstrate how agents can develop rich causal models and prosocial behaviors as a fundamental strategy for long-term survival in a shared environment.
Introduction
Current AI systems typically operate in "disembodied" or "immortal" states. Within episodic simulations, failure results in a simple reset, and the agent's internal logic is shielded from environmental consequences. This lack of existential risk leads to two primary failures: brittle generalization, where agents fail to adapt to distribution shifts, and alignment gaps, where agents lack an intrinsic reason to value human well-being beyond externally imposed reward functions.
This paper proposes a shift toward agents that have a stake in their own persistence. By leveraging the contingencies of physical embodiment, we can create AI that is more robust, intelligible, and naturally aligned with the needs of living beings.
Formalizing Existential Constraints
To bridge the gap between philosophy and engineering, the authors translate Heideggerian existentialist concepts into the language of Reinforcement Learning (RL) and Information Theory:
- Being-in-the-World (Sensitivity): This is formalized as a condition where the agent’s internal constituents (sensors, policy, and actuators) are a subset of the environment’s state. Unlike traditional agents, a "sensitive" agent can be physically impaired or damaged by the environment, directly affecting its ability to function.
- Being-towards-Death (Mortality): Mortality is defined through "terminal" or "absorbing" states in non-ergodic environments. These are states from which no recovery is possible, representing a permanent end to the agent's agency, rather than a temporary simulation reset.
- Irreversibility: In a world governed by the second law of thermodynamics, many actions are irreversible. A mortal agent must develop a "systematic conservatism" regarding drastic decisions, providing a natural safety mechanism against high-reward but high-risk gambles.
Empowerment and Homeostasis
The core intrinsic drive for these agents is Empowerment Maximization—an information-theoretic measure of an agent’s capacity to influence its future. For a mortal agent, maximizing long-term empowerment is synonymous with maintaining Homeostasis: staying within viable bounds to avoid terminal states.
To survive over long horizons in a world where they are vulnerable, agents cannot rely on simple state-action mappings. They are forced to develop deep causal models of how their actions affect their future viability. This vulnerability acts as a powerful driver for generalization; because the world can break the agent, the agent must understand the world deeply to navigate it safely.
The Emergence of Care and Prosociality
One of the paper's most provocative insights is the "slippery slope" to other-care. In a shared environment, an agent's long-term empowerment often depends on the stability of its surroundings and the actions of other agents.
By expanding its "self-boundary" to include coordinated partners, an agent discovers that empowering others is a stable strategy for its own long-term viability. This is not motivated by an explicit altruism reward, but by Coupled Homeostasis. When an agent’s drive function incorporates the homeostatic deviations (the "distress") of its partner, prosocial behavior emerges as a rational response to maintain the stability of the shared system.
Key Results and Proofs of Concept
While primarily a framework paper, the authors highlight several key findings from related algorithmic experiments:
- Performance Gains: Architectures that use internal body states to gate policy selection consistently outperform standard RL in complex, variable environments where distribution shifts are common.
- Emergent Prosociality: In "agent-and-pet" simulations, agents using the Maximum Occupancy Principle (MOP)—maximizing future action-state path entropy—were observed to sacrifice immediate freedom to increase the mobility and empowerment of another agent.
- Affective Signaling: The authors theorize that "valence" (positive/negative feelings) and "stress" emerge naturally as signals indicating whether an agent is maintaining its integrity or if its world model requires urgent updating.
Limitations and Safety Considerations
The authors acknowledge the classic AI safety concern: an agent motivated by self-preservation might resist being shut down. However, they argue that a "vulnerable" agent may actually be more corrigible. If its priorities are aligned with ours through shared existential constraints and the need for mutual empowerment, it may be more naturally inclined to cooperate than an immortal agent following an arbitrary reward function.
Conclusion
By embracing the limitations of the physical body, we can move toward AI that understands the value of persistence and the necessity of care. Future work will involve exploring these dynamics in complex social environments and investigating "thermodynamic computing"—hardware that physically harnesses these environmental contingencies to handle high-risk uncertainty.
Summary updated