Abstract
Visual localization in changing environments remains a significant challenge for autonomous systems. This paper presents a novel image-retrieval architecture that utilizes condition-specific sub-networks and Generalized-Mean (GeM) pooling to bridge the domain gap between disparate environmental conditions. By specializing network branches for specific conditions like night or rain, the proposed method achieves a nearly 300% improvement in nighttime localization accuracy on the RobotCar Seasons benchmark compared to traditional VLAD-based approaches, without increasing inference time.
Introduction: The Challenge of Cross-Condition Localization
Predicting a camera's 6-degree-of-freedom (6 DoF) pose is essential for autonomous driving and augmented reality. While feature-based methods (e.g., SIFT) and retrieval-based methods (e.g., NetVLAD) perform well under consistent lighting, they often fail when faced with drastic appearance changes, such as the transition from day to night. Local features often lack the repeatability required to match daytime landmarks under nighttime illumination, and global descriptors can be overwhelmed by the domain shift between a sunny afternoon and a rainy midnight.
Methodology
The authors propose a Siamese network architecture designed to learn condition-robust representations. The framework consists of several key innovations:
Condition-Specific Sub-networks
Rather than forcing a single network to generalize across all environments, the architecture employs "expert" branches ($f_c$) specialized for different conditions (e.g., sun, rain, night). During processing, an image is routed through the branch corresponding to its captured condition.
Condition-Agnostic Integration
Following the specialized branches, features pass through shared condition-agnostic blocks ($g$). These layers map the specialized features into a unified representation space, enabling direct comparison between a nighttime query and a daytime database entry.
Generalized-Mean (GeM) Pooling
The architecture replaces complex VLAD or NetVLAD layers with GeM pooling. GeM acts as a learnable interpolation between average and max pooling, providing a more discriminative and robust global descriptor for image retrieval tasks.
Training Strategy
To ensure balanced performance, the model is trained using a condition-balanced sampling strategy. This prevents the network from biasing toward more frequent conditions (like clear daylight) and ensures the "expert" branches are equally well-tuned.
Evaluation and Results
The method was evaluated on the RobotCar Seasons dataset, which contains over 20,000 daytime reference images and 2,600 nighttime query images captured over a year in Oxford.
Performance Gains
The proposed approach demonstrated a massive performance leap, improving nighttime localization results by approximately 300% over popular VLAD-based methods. It outperformed state-of-the-art retrieval methods like NetVLAD and DenseVLAD, as well as feature-based methods like Active Search, particularly at coarse precision levels (5m, 10°).
Efficiency
A critical advantage of this multi-branch approach is that only one branch is active during the forward pass. Consequently, the performance gains are achieved with zero additional inference overhead compared to standard single-branch pipelines. Ablation studies further revealed that performance on nighttime images improved as more blocks in the ResNet-50 backbone were made condition-specific.
Conclusion and Future Work
While the condition-based representation significantly improves robustness, challenges remain in cases of extreme occlusion or near-total darkness. Future research directions include integrating this retrieval stage with local feature alignment for centimeter-level precision and utilizing GAN-generated synthetic data to augment training sets for rare environmental conditions.
Summary updated