Abstract
Visual place recognition is challenging, especially when only a few place exemplars are given. To mitigate the challenge, we consider place recognition method using omnidirectional cameras and propose a novel Omnidirectional Convolutional Neural Network (O-CNN) to handle severe camera pose variation. Given a visual input, the task of the O-CNN is not to retrieve the matched place exemplar, but to retrieve the closest place exemplar and estimate the relative distance between the input and the closest place. With the ability to estimate relative distance, a heuristic policy is proposed to navigate a robot to the retrieved closest place. Note that the network is designed to take advantage of the omnidirectional view by incorporating circular padding and rotation invariance. To train a powerful O-CNN, we build a virtual world for training on a large scale. We also propose a continuous lifted structured feature embedding loss to learn the concept of distance efficiently. Finally, our experimental results confirm that our method achieves state-of-the-art accuracy and speed with both the virtual world and real-world datasets.
Introduction
Visual Place Recognition (VPR) is the fundamental "Where am I?" problem for autonomous robots. Traditional systems often rely on dense maps of images; if a robot is standing exactly where a map photo was taken, it can localize itself. However, if the map is sparse—containing only a few photos of a large environment—and the robot is standing far from any exemplar, standard methods often fail.
This paper introduces the Omnidirectional CNN (O-CNN), a deep learning framework designed to help robots recognize their location and navigate even with minimal map data. By leveraging 360-degree cameras, the system doesn't just find an exact match for a location; it estimates the physical distance to the nearest known point and uses that information to guide the robot toward its target.
Methodology
The O-CNN framework utilizes a GoogleNet backbone modified with three specific innovations tailored for omnidirectional (360°) imagery:
- Circular Padding: Standard CNNs pad image edges with zeros, creating artificial boundaries. Since 360° images "wrap around," O-CNN uses circular padding to ensure the network understands that the far-left and far-right of the image are spatially connected.
- Roll Branching: To handle rotational variation (e.g., a robot facing a different direction than the map exemplar), the network creates multiple "branches" of shifted features. This allows the model to find the optimal rotational alignment between the current view and the map.
- Continuous Lifted Structured Feature Embedding: Traditional metric learning treats place recognition as a binary classification. This novel loss function instead teaches the network to arrange features such that the distance in "feature space" directly corresponds to the physical distance in the real world (meters).
Once the O-CNN identifies the closest exemplar and the distance to it, a Local Gradient-based Heuristic Policy is employed. The robot performs a local search to determine which direction reduces the estimated distance most effectively, essentially "feeling" its way toward the target location.
Experimental Results
To train and validate the O-CNN, the authors developed a Virtual World Dataset using Unity 3D, featuring 15 diverse indoor scenes. This was supplemented by a Real-World Dataset collected using a LUNA 360 camera in residential and laboratory environments.
Performance Metrics
- Accuracy: In virtual environments, O-CNN achieved a 69.7% average recall across different error tolerances, significantly outperforming baselines like NetVLAD (43.8%) and Disloc (62.0%).
- Speed: The system is highly efficient, with an inference speed of 128.21 ms per query, compared to over 2400 ms for SIFT-based methods.
- Navigation Success: In navigation tests, the O-CNN-based policy reached the target with an 84% success rate, requiring fewer steps (avg. 22) than the NetVLAD baseline (68% success, avg. 36 steps).
Conclusion and Future Work
The O-CNN demonstrates that treating place recognition as a continuous distance-ranking problem, rather than simple classification, significantly improves robot localization in sparse environments. While the current heuristic policy is effective, future work may involve integrating these features with advanced path planning or Reinforcement Learning. Additionally, scaling the retrieval process for massive environments remains an area for further optimization.
Summary updated