Adaptive Modeling for Risk-Aware Decision Making

Sandhya SaisubramanianAAAI 2019 · DOI

Introduction

Autonomous agents operating in dynamic, real-world environments face a fundamental tension between computational efficiency and safety. To make decisions in real-time, these agents often rely on "reduced models"—simplified versions of the world that ignore certain outcomes to conserve processing power. However, this simplification often leads to the "optimism trap": by pruning unlikely or complex branches of a decision tree, an agent may inadvertently ignore rare but catastrophic risks, such as a collision or a mechanical failure. This research proposes an adaptive modeling framework designed to balance these competing needs by dynamically adjusting model fidelity based on environmental risk.

The Problem: The Optimism Trap

Most autonomous agents reason using Markov Decision Processes (MDPs) or Stochastic Shortest Path (SSP) models. When these models are simplified for speed, they become "blind" to the very tail risks that lead to disaster. The author identifies three primary hurdles in current planning paradigms:

  1. Context-Specific Risk: Risk is often problem-specific, making it difficult to design a universal safe model.
  2. Computational Complexity: Increasing model detail (fidelity) leads to exponential increases in mathematical complexity and processing time.
  3. Information Scarcity: Agents frequently lack complete information about their environment until they are already in motion.

Proposed Methodology: Adaptive Modeling

To address these challenges, this research shifts away from static, single-model planning toward a dynamic Adaptive Modeling approach. The framework consists of several key components:

  • Portfolio of Reduced Models: Instead of relying on one simplified model, the agent maintains a collection of models with varying levels of detail.
  • Selective Fidelity: The agent "zooms in" on high-fidelity details only in specific states where risk is high, while maintaining simpler models for safe, predictable areas. This treats model accuracy as a finite resource to be allocated where it matters most.
  • Cost Adjustments: To account for the details ignored in reduced models, the framework adjusts the "cost" of actions. This serves as a heuristic to prevent the agent from becoming overly optimistic about paths that appear safe only because their risks are hidden.
  • Negative Side Effect Mitigation: The framework specifically addresses "unsafe replanning." In many scenarios, the act of stopping to re-calculate a route can be dangerous—for example, a robot should not pause to replan while balancing in a precarious position or navigating a busy intersection.

Preliminary Results and Insights

Testing on Stochastic Shortest Path (SSP) problems and autonomous navigation scenarios has yielded several promising findings:

  • Efficiency and Safety: The portfolio approach achieves a significant reduction in negative side effects without the massive computational overhead required by full-fidelity models.
  • Bounded Optimality: The research identifies conditions under which an agent can remain "boundedly optimal," finding the best possible solution within its computational limits without sacrificing safety.
  • Replanning as Risk: A key insight of this work is that replanning is not always a safety net; in physical systems, the computational delay of replanning can itself be a risk factor.

Future Directions

The current framework assumes the agent has access to full information, even if it chooses to ignore it for speed. Future work will expand this to:

  • Incomplete Information: Developing methods for agents to query an "oracle" or use abstractions when risks are unknown.
  • Automated Risk Discovery: Utilizing machine learning (such as regression and decision stumps) to automatically identify environmental features that represent risks, reducing the need for human-defined safety parameters.
  • Side Effect Classification: Refining the system to distinguish between acceptable trade-offs and truly dangerous side effects.

Summary updated

BibTeX
@inproceedings{saisubramanian2019adaptive,
  author = {Sandhya Saisubramanian},
  title = {{Adaptive Modeling for Risk-Aware Decision Making}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.1609/aaai.v33i01.33019896},
}

Loading related content...

Figure