Hallucinating Robots: Inferring Obstacle Distances from Partial Laser Measurements

Jens Lundell, Francesco Verdoja, Ville KyrkiIROS 2018 · DOI

TL;DR

This paper introduces a novel deep learning approach for mobile robots to "hallucinate" obstacle distances from limited 2D laser scanner data. By training an autoencoder with skip connections and a specialized loss function, the robot can infer distances to otherwise invisible obstacles like glass or tabletops, enhancing safety and navigation capabilities.

What Problem Does This Paper Address?

Mobile robots heavily rely on 2D laser scanners for navigation, mapping, and localization due to their efficiency and wide field of view. However, these sensors have a critical limitation: they cannot detect obstacles that are transparent (like glass) or whose relevant parts are not at the sensor's height (like tabletops). This can lead to dangerous situations where a robot plans a path through an unseen obstacle, potentially causing a collision. While richer sensors like 3D lasers or RGBD cameras can overcome this, they often come with higher computational costs and limited fields of view. This paper addresses the challenge of inferring these missing obstacle distances using only the raw 2D laser data, aiming to enrich the robot's environmental awareness without requiring additional, more complex sensors.

Methods

The core of the proposed method is a fully convolutional autoencoder architecture. The autoencoder is trained to map raw 2D laser scan data (input) to a predicted obstacle distance vector (output).

Key novelties and techniques include:

  • Autoencoder with Skip Connections: Inspired by image segmentation networks, skip connections are introduced between corresponding layers of the encoder and decoder. This allows high-frequency details, which might be lost during the encoding process, to be directly passed to the decoder, enabling a more precise reconstruction of obstacle distances.
  • Non-uniform Scaling (γ-scaling): To prioritize safety, errors in distance estimation for closer objects are more critical than for distant ones. The authors propose scaling the output of the network by a factor γ. When γ > 1, this effectively increases the resolution in the near-range portion of the output, allowing for more accurate estimations of nearby obstacles.
  • Root Mean Squared Logarithmic Error (RMSLE) Loss Function: This loss function is chosen to complement the γ-scaling. RMSLE is particularly useful in domains where large errors for large numbers are less critical than smaller errors for smaller numbers, aligning with the safety priority of accurate near-range obstacle detection.
  • Data Augmentation: The training data is augmented by adding random Gaussian noise to the input laser scans and by randomly flipping input-output pairs. This helps improve the network's generalization capabilities and robustness.

Datasets

The authors created a custom dataset by teleoperating the Care-O-bot 4 robot in a university building.

  • Training Set: Consists of 27,904 input-output pairs collected from the second floor.
  • Test Set: Consists of 11,604 input-output pairs collected from the third floor.

The ground truth data (y) was generated by fusing raw 2D laser scans (x) with depth images from RGBD cameras. Specifically, depth images were projected to 1D, and the closest distance from either the laser scanner or the depth camera was taken as the ground truth. This method avoids manual labeling and captures challenging scenarios with transparent or partially visible obstacles.

Key Insights and Ideas

  • "Hallucinating" is a Feature, Not a Bug: The paper reframes the network's ability to infer unseen data as a valuable feature for enhancing robot perception, rather than a limitation of the sensor.
  • Prioritizing Near-Range Accuracy: The γ-scaling and RMSLE loss function demonstrate a sophisticated understanding of robot safety, where accurate distance estimation for nearby objects is paramount.
  • Skip Connections are Crucial for Detail: The quantitative results strongly suggest that skip connections are vital for the network to effectively pass and utilize fine-grained information for accurate distance inference.
  • Noise Injection Improves Robustness: While noise injection might slightly degrade average performance, it significantly reduces the variance in performance, leading to more consistent and reliable results.

Key Results

  • Quantitative Performance: The best-performing network configuration, featuring skip connections, γ=2, and Gaussian noise (σ=0.02), achieved the lowest RMSLE on the test set. This configuration significantly outperformed a baseline without these enhancements.
  • Qualitative Performance: Online testing on the Care-O-bot 4 robot showed impressive results in real-time. The robot successfully inferred obstacle distances in challenging scenarios, including rooms with glass walls, tables blocking doors, and even from sparse laser points.
  • 360° Coverage: The network, trained on 90° views, was able to hallucinate obstacle distances for a full 360° around the robot by processing laser data in chunks.
  • Improved Navigation: When used with a local planner, the hallucinated laser data enabled the robot to successfully navigate through a doorway blocked by a table, avoiding a collision that a planner using only raw laser data would have encountered.

Limitations and Future Work

  • Generalization to Novel Scenarios: The authors acknowledge that the network struggled with scenarios significantly different from the training set, such as objects placed perpendicularly to walls or isolated in empty spaces.
  • Smoothness Tendency: The network tends to produce smooth curves, which can be problematic for navigation planning through doorways, potentially making them appear unreachable.
  • Future Work:
    • Domain Randomization: Augmenting the training data with randomized object positions and rotations to improve generalization.
  • Temporal Component: Incorporating a temporal component into the network to handle moving obstacles.
  • End-to-End Integration: Integrating the hallucinated laser data directly into motion planning for autonomous navigation.
  • Addressing Smoothness: Exploring network topologies that enforce stronger non-linearities to mitigate the smoothing effect.

Summary updated

BibTeX
@inproceedings{lundell2018hallucinating,
  author = {Jens Lundell and Francesco Verdoja and Ville Kyrki},
  title = {{Hallucinating Robots: Inferring Obstacle Distances from Partial Laser Measurements}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2018},
  doi = {10.1109/iros.2018.8594399},
}
Figure