CRA: Continuous Risk Assessment for Safety-critical Visual Navigation

Lihao Zheng, Ying Wu, Chen Jiang, Randy FreemanarXiv 2026 · arXiv

In the world of autonomous navigation, robots have traditionally been "semantically blind." To a standard safety filter, a cardboard box and a running child are both just geometric obstacles to be avoided. But in the real world, safety isn't binary; it’s a continuous spectrum influenced by speed, heat, and context.

Researchers from Northwestern University recently presented CRA (Continuous Risk Assessment), a framework that allows robots to "see" danger more like humans do—by assessing physical properties and adjusting their behavior accordingly.

TL;DR

CRA is a vision-based framework that replaces binary "safe/unsafe" zones with a continuous risk score based on four physical factors: kinetic, potential, contact-severity, and thermal risk. By integrating these scores into a real-time safety filter, robots can perform more nuanced, context-aware detours in complex environments like busy restaurants.

What Problem Does This Paper Address?

Most safety-critical control methods rely on predefined geometric constraints. If a robot is $X$ centimeters away from an object, it’s "safe"; any closer, and it’s "unsafe." While recent work has added semantic labels (e.g., "this is a person"), these are still often treated as discrete rules.

The authors argue that real-world risk is continuous. A person carrying a heavy, hot tray at high speed poses a significantly higher risk than a static chair, even if they occupy the same amount of space. Furthermore, using massive Foundation Models (like LLMs or VLMs) to reason about these risks in real-time is often too slow for high-speed navigation. CRA bridges this gap by providing a low-latency way to turn visual reasoning into immediate control actions.

Figure 1. A running child approaches the robot. The robot “sees” the danger, and its controller responds immediately through the safety filter.

Methods

The CRA framework operates in three main stages:

  1. RGB-LiDAR Association: The system uses SAM 2.1 and YOLO11n-seg to segment objects and humans in RGB images. It then projects 3D LiDAR points onto these masks to associate geometric data with visual instances.
  2. Four-Factor Risk Assessment: For every detected object, the system estimates:
  3. Kinetic Risk: Combines estimated mass (via a MobileNetV3-based regressor) with relative velocity.
  4. Potential Risk: Assesses the risk of falling (e.g., a hole) or objects falling from heights.
  5. Contact-Severity: Uses corner-detection and specialized edge-segmentation to identify "sharp" vs. "blunt" objects.
  6. Thermal Risk: Detects visual proxies for heat, such as steam, smoke, or flames, using SAM3.
  7. Safety Filter Integration: These factors are aggregated into a single risk score $\phi$. This score modulates the "repulsive" strength of a Laplace Guidance Field (LGF), which in turn shapes a Poisson Safety Function (PSF). Finally, a Model Predictive Control (MPC) layer uses these fields to adjust the robot's path in real-time.
Figure 2. Overview of the CRA framework. RGB–LiDAR association provides object evidence for four-factor risk estimation. The resulting per-object risk scores condition Safe-SAGE navigation.

Datasets

The authors utilized several datasets to train and evaluate specific components of the pipeline:

  • ABO (Amazon Berkeley Objects): Used to train the mass estimation model on product images and catalog weights.
  • PACO (Parts and Attributes of Common Objects): Used to evaluate contact risk and fine-tune YOLO for detecting exposed sharp edges (like knife blades).
  • Pexels: Real-world video clips of fire and steam were used to validate the thermal risk assessment.
  • NVIDIA Isaac Sim: The entire system was evaluated in a simulated restaurant environment using the Omniverse Restaurant Demo assets.

Key Insights and Ideas

  • Risk as a Spectrum: The core "aha moment" is moving from "Is this an obstacle?" to "How dangerous is this interaction?" This allows for "soft" detours around low-risk items and "hard" interventions for high-risk ones.
  • Physical Reasoning from Pixels: By estimating mass and heat from visual cues, the robot gains a "physical intuition" that geometry alone cannot provide.
  • Decoupling Reasoning from Control: By mapping complex visual reasoning to a simple boundary flux parameter in a potential field, the authors can use fast, classical control methods (MPC/CBF) that provide rigorous safety guarantees without the latency of a large neural network in the loop.

Key Results

The system was tested across various scenarios in simulation:

  • Contact Risk: Sharp objects (knives, scissors) received a mean risk score of 0.709, while rounded/blunt objects averaged 0.576, leading the robot to give sharp objects a wider berth.
  • Thermal Risk: The system correctly ranked hazards, assigning higher risk to wood fires (0.813) and gas burners (0.791) than to steaming pots (0.738).
  • Potential Risk: In a "hole avoidance" test, the CRA-enabled robot successfully detoured around a shallow pit that a baseline geometric filter (which only saw the floor) failed to detect, causing the baseline robot to fall in.
  • Kinetic Risk: As the robot's requested speed increased, the risk score for nearby obstacles rose dynamically, forcing larger, safer detours at higher speeds.
Visual results for the four risk factors, showing how different physical properties result in different risk scores and robot trajectories.

Limitations and Future Work

While impressive, the authors note several areas for improvement:

  • Visual Proxies: Thermal risk currently relies on visible steam or smoke. The system cannot "see" a hot surface that isn't off-gassing.
  • Mass Estimation: The current mass estimation is a "rough estimate" and could be improved with more sophisticated multi-view reasoning.
  • Real-World Deployment: The current study is limited to high-fidelity simulation (Isaac Sim). Future work will focus on deploying CRA on physical robots in real-world social environments to test its performance under varying lighting and sensor noise.

Summary updated

Abstract

Traditional safety-critical control methods often rely on predefined constraints that separate states into safe and unsafe regions. While recent studies have extended geometric safety toward semantic safety, real-world safety is rarely purely binary. The risk associated with an object or human can vary continuously with its physical properties, like motion, and interaction context; for example, a person carrying a heavy object or a hot object moving at high speed may require a different safety response from an ordinary static obstacle. In this paper, we propose a vision-based continuous risk assessment framework for safety-critical visual navigation. Our method uses RGB and LiDAR observations to estimate four risk factors (kinetic, potential, contact-severity, and thermal) and combines them into a continuous risk score for each assessed object or person. These estimates are integrated into a Poisson safety function with a Laplace guidance field as a safety filter, enabling online monitoring and intervention in open-world robot navigation. We demonstrate the proposed method in a simulated restaurant scene.
BibTeX
@misc{zheng2026cra,
  author = {Lihao Zheng and Ying Wu and Chen Jiang and Randy Freeman},
  title = {{CRA: Continuous Risk Assessment for Safety-critical Visual Navigation}},
  year = {2026},
  eprint = {2611.22080},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.22080},
}
Figure