LARGER RECEPTIVE FIELD BASED RGB VISUAL RELOCALIZATION METHOD USING CONVOLUTIONAL NETWORK

Jiangying Qin, Min Li, D. Li, Xuan Liao, Jiageng Zhong, Han Zhang2021 · DOI

Abstract

Visual Relocalization is a key technology in many computer vision applications. Traditional visual relocalization is mainly achieved through geometric methods, while PoseNet introduces convolutional neural network in visual relocalization for the first time to realize real-time camera pose estimation based on a single image. Aiming at the problem of accuracy and robustness of the current PoseNet algorithm in complex environment, this paper proposes and implements a new high-precision robust camera pose calculation method (LRF-PoseNet). This method directly adjusts the size of the input image without cropping, so as to increase the receptive field of the training image. Then, the image and the corresponding pose tags are input into the improved LSTM-based PoseNet network for training, and the Adam optimizer is used to optimize the network. Finally, the trained network is used to estimate the camera pose. Experimental results on open RGB dataset show that the proposed method in this paper can obtain more accurate camera pose compared with the existing CNN-based methods.

Introduction

Visual relocalization—the ability of a system to determine its exact position and orientation (6-DOF pose) from a single image—is a cornerstone of robotics, autonomous driving, and augmented reality. Traditional methods rely on geometric feature matching (like SIFT or ORB), which often fail in environments with motion blur, poor lighting, or few textures. While deep learning models like PoseNet solved some of these robustness issues by treating relocalization as a regression problem, they introduced new challenges:

  1. Information Loss: Standard PoseNet crops the center of an image to fit a fixed input size, discarding peripheral data that could be vital for localization.
  2. Overfitting: The high-dimensional output of fully connected layers in these networks often leads to overfitting, especially when training data is limited.
  3. Suboptimal Optimization: Relying on standard Stochastic Gradient Descent (SGD) can lead to slow convergence or getting stuck in local optima.

Methodology: LRF-PoseNet

The proposed LRF-PoseNet (Larger Receptive Field PoseNet) introduces three key modifications to the standard PoseNet pipeline to address these issues.

Full-View Resizing

Instead of scaling and cropping the center of an image (the standard GoogleNet approach), LRF-PoseNet resizes the entire image to $224 \times 224$ pixels. While this changes the aspect ratio, it ensures the network "sees" the entire scene, significantly increasing the receptive field.

Comparison of full-view vs. cropped regions. The red boxes in (b) highlight peripheral information typically lost in standard cropping methods.

Multi-Directional LSTM

To combat overfitting in the high-dimensional (2048D) feature space, the feature vector is reshaped into a $32 \times 64$ matrix and passed through four Long Short-Term Memory (LSTM) units scanning up, down, left, and right. This "structured dimensionality reduction" helps the network learn spatial dependencies and select the most useful feature correlations across the image.

Adam Optimizer

The traditional SGD is replaced with the Adam optimizer, which uses adaptive learning rates for different parameters, leading to more stable and faster training.

Experimental Results

The model was evaluated on two well-known public benchmarks: the CoRBS Dataset (563 training images, 48 test images) and the TUM RGB-D Benchmark Dataset (1,197 training images, 325 test images).

Dataset Metric PoseNet LRF-PoseNet Improvement
CoRBS Position Error (m) 0.48m 0.04m ~91%
Orientation Error (°) 5.08° 2.44° ~52%
TUM Position Error (m) 0.35m 0.21m ~40%
Orientation Error (°) 4.16° 3.62° ~13%

On the CoRBS dataset, LRF-PoseNet achieved a staggering 79% of test points within a 0.2m error range, whereas the original PoseNet had zero points in that range.

Discussion and Insights

The results suggest that Receptive Field is more critical than Image Resolution. Preserving the full field of view is more important than maintaining the original aspect ratio. Even though resizing distorts the image, the network benefits more from seeing peripheral landmarks that would otherwise be cropped out.

Furthermore, using LSTMs as Spatial Filters allows the network to "scan" a static feature map and understand how features in one part of the image relate to features in another. This effectively acts as a sophisticated spatial filter that reduces noise and prevents overfitting.

Conclusion

LRF-PoseNet significantly improves camera relocalization accuracy by rethinking image input strategies and feature correlation. While the method demonstrates significant improvements, future work will explore maintaining aspect ratio consistency and comparing performance against hybrid methods that combine deep learning with geometric constraints.

Summary updated

Abstract

Visual Relocalization is a key technology in many computer vision applications. Traditional visual relocalization is mainly achieved through geometric methods, while PoseNet introduces convolutional neural network in visual relocalization for the first time to realize real-time camera pose estimation based on a single image. Aiming at the problem of accuracy and robustness of the current PoseNet algorithm in complex environment, this paper proposes and implements a new high-precision robust camera pose calculation method (LRF-PoseNet). This method directly adjusts the size of the input image without cropping, so as to increase the receptive field of the training image. Then, the image and the corresponding pose tags are input into the improved LSTM-based PoseNet network for training, and the Adam optimizer is used to optimize the network. Finally, the trained network is used to estimate the camera pose. Experimental results on open RGB dataset show that the proposed method in this paper can obtain more accurate camera pose compared with the existing CNN-based methods.
BibTeX
@inproceedings{qin2021larger,
  author = {Jiangying Qin and Min Li and D. Li and Xuan Liao and Jiageng Zhong and Han Zhang},
  title = {{LARGER RECEPTIVE FIELD BASED RGB VISUAL RELOCALIZATION METHOD USING CONVOLUTIONAL NETWORK}},
  booktitle = {˜The},
  year = {2021},
  doi = {10.5194/isprs-archives-xliii-b2-2021-47-2021},
}
Figure