An Attentional Recurrent Neural Network for Occlusion-Aware Proactive Anomaly Detection in Field Robot Navigation

Andre Schreiber, Tianchen Ji, D. Livingston McPherson, Katherine Rose Driggs-Campbell

Abstract

The use of mobile robots in unstructured environments like the agricultural field is becoming increasingly common. The ability for such field robots to proactively identify and avoid failures is thus crucial for ensuring efficiency and avoiding damage. However, the cluttered field environment introduces various sources of noise (such as sensor occlusions) that make proactive anomaly detection difficult. Existing approaches can show poor performance in sensor occlusion scenarios as they typically do not explicitly model occlusions and only leverage current sensory inputs. In this work, we present an attention-based recurrent neural network architecture for proactive anomaly detection that fuses current sensory inputs and planned control actions with a latent representation of prior robot state. We enhance our model with an explicitly-learned model of sensor occlusion that is used to modulate the use of our latent representation of prior robot state. Our method shows improved anomaly detection performance and enables mobile field robots to display increased resilience to predicting false positives regarding navigation failure during periods of sensor occlusion, particularly in cases where all sensors are briefly occluded.

Introduction

Navigating a cornfield is a significant challenge for autonomous robots. Unlike the structured environments of warehouses, agricultural fields are filled with "catastrophic noise": low-hanging leaves that slap cameras, dust that obscures LiDAR, and uneven terrain. This paper focuses on Proactive Anomaly Detection (AD)—the robot's ability to analyze current sensor data and planned paths to predict potential failures, such as crashing or leaving a row, within a short time horizon.

A major hurdle for current proactive models is their "forgetfulness"; they often rely solely on the current frame of data. If a leaf briefly covers the camera and LiDAR simultaneously, these models often report a failure (a false positive), causing the robot to stop unnecessarily. To achieve true autonomy, robots must differentiate between temporary sensor blackouts and genuine obstacles.

Examples of sensor data in a cornfield environment, showing camera occlusion (a) and dense stalks (b).

Methodology: The ROAR Architecture

The researchers introduced ROAR (Recurrent Occlusion-AwaRe), a neural network architecture designed to maintain a temporal "memory" of the robot's state. Its design centers on three main components:

  1. Multi-Modal Fusion: It processes three distinct inputs: 2D LiDAR scans, RGB camera images, and the robot’s planned trajectory (rendered as a grayscale path image).
  2. Explicit Occlusion Modeling: ROAR utilizes specific sub-networks trained to predict whether the camera or LiDAR is currently occluded. This gives the model "meta-awareness" of its own sensor reliability.
  3. Attentional Recurrent State: The system maintains a latent state ($h^{(t)}$) that evolves over time. An attention mechanism fuses incoming data, with the query for the robot's internal memory biased by occlusion predictions. If the sensors report being blocked, the mechanism shifts focus toward the internal memory of the clear path seen previously, rather than the noisy, occluded input.

Experimental Results

The study utilized a dataset collected via the TerraSentia mobile robot, involving 4.1 km of navigation through corn rows. The dataset was manually labeled for both navigation failures and sensor occlusions.

Key results include:

  • Superior Accuracy: ROAR achieved a Precision-Recall Area Under Curve (PR-AUC) of 0.800, outperforming baselines like PAAD (0.790) and Graph Fusion (0.775).
  • Robustness to Total Occlusion: In tests where both LiDAR and camera were blocked, ROAR maintained a low probability of failure, correctly identifying the situation as a temporary glitch, while previous models' failure predictions spiked immediately.
  • Real-Time Performance: Despite the complexity of recurrence and attention, ROAR maintains an inference time of approximately 4.99 ms, making it suitable for real-time deployment on edge hardware.

Conclusion and Future Work

ROAR demonstrates that memory and explicit occlusion modeling are vital for reliable field robot navigation. By "remembering" that a path was clear moments before a sensor blackout, the robot can avoid unnecessary emergency stops. Future work will focus on moving toward semi-supervised or unsupervised learning to reduce the reliance on labor-intensive human-annotated labels for failures and occlusions.

Summary updated

Abstract

The use of mobile robots in unstructured environments like the agricultural field is becoming increasingly common. The ability for such field robots to proactively identify and avoid failures is thus crucial for ensuring efficiency and avoiding damage. However, the cluttered field environment introduces various sources of noise (such as sensor occlusions) that make proactive anomaly detection difficult. Existing approaches can show poor performance in sensor occlusion scenarios as they typically do not explicitly model occlusions and only leverage current sensory inputs. In this work, we present an attention-based recurrent neural network architecture for proactive anomaly detection that fuses current sensory inputs and planned control actions with a latent representation of prior robot state. We enhance our model with an explicitly-learned model of sensor occlusion that is used to modulate the use of our latent representation of prior robot state. Our method shows improved anomaly detection performance and enables mobile field robots to display increased resilience to predicting false positives regarding navigation failure during periods of sensor occlusion, particularly in cases where all sensors are briefly occluded. Our code is available at: https://github.com/andreschreiber/roar
BibTeX
@inproceedings{schreiber2023attentional,
  author = {Andre Schreiber and Tianchen Ji and D. Livingston McPherson and Katherine Rose Driggs-Campbell},
  title = {{An Attentional Recurrent Neural Network for Occlusion-Aware Proactive Anomaly Detection in Field Robot Navigation}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2023},
  doi = {10.1109/iros55552.2023.10341852},
}

Loading related content...

Figure