Abstract
In autonomous navigation and augmented reality, camera relocalisation—determining a device's 6-DoF pose from a single image—is a critical task. Traditional geometric maps scale linearly with environment size, posing memory challenges for mobile agents. This paper investigates using Convolutional Neural Networks (CNNs) as compact, constant-size world representations. By training various VGG architectures to regress poses, the authors demonstrate that network weights can effectively store spatial information, with shallower models performing comparably to deeper ones as training data increases, offering a viable path for map compression.
Introduction
The "relocalisation" problem involves estimating the position and orientation of a camera within a known environment. Conventional methods rely on point clouds or feature maps, which grow in size as more area is mapped. Researchers at the University of Bristol propose a paradigm shift: treating the CNN itself as the map. In this approach, the environment is encoded within the fixed-size parameters of a neural network, decoupling map memory requirements from the physical extent of the scene.
Methodology
The authors frame relocalisation as a 6-Degree-of-Freedom (6-DoF) regression task. Given an input image $I$, the network predicts a pose $p = [x, q]$, where $x$ represents 3D position and $q$ represents orientation (quaternion).
Architectures and Transfer Learning
The study evaluates five VGG-based models, ranging from VGG-F (8 layers, 61M parameters) to VGG-19 (19 layers, 144M parameters). To overcome the data-hungry nature of deep learning, the models utilize transfer learning from ImageNet. This allows the networks to leverage low-level visual features (edges, textures) learned from millions of generic images, focusing their training on scene-specific spatial regression.
Loss Function
The training employs the PoseNet loss function, which balances Euclidean distance for position and quaternion distance for orientation using a scale factor ($\beta$).
Experimental Setup
The models were benchmarked on three diverse datasets:
- St Marys Church: A large outdoor scene with nearly 1,500 training frames.
- TUM Long Office Sequence: A textured indoor environment.
- Microsoft 7-Scenes: A standard benchmark for indoor relocalisation featuring varied trajectories recorded by different users.
Results and Discussion
The core finding is that CNN-based maps offer significant compression without drastic loss in accuracy. On the "Red Kitchen" dataset, the 8-layer VGG-F achieved a mean position error of 0.559m, which is competitive with the 23-layer PoseNet (0.59m).
As shown in the results, the performance gap between shallow (VGG-F) and deep (VGG-19) models narrows as the volume of training data increases. This suggests that for a fixed environment, a smaller network can reach a performance plateau similar to larger models, effectively compressing the spatial representation into fewer weights. Furthermore, pre-trained RGB models consistently outperformed Depth or Point Cloud inputs, indicating that ImageNet pre-training provides a superior feature foundation for spatial regression.
Conclusion
This work demonstrates that CNNs can serve as efficient, constant-size representations of physical environments. While they currently lack the millimeter-level precision of state-of-the-art geometric SLAM systems, their ability to compress complex maps into a fixed memory footprint is a significant advantage for resource-constrained mobile agents. Future research will focus on integrating semantic context to further enhance relocalisation robustness.
Summary updated