To know exactly where a device is in six degrees of freedom (6-DoF), traditional structure-based localization algorithms match local image features against an explicit 3D point cloud. However, as Thomas Schneider et al. note in maplab: An Open Framework for Research in Visual-inertial Mapping and Localization, continuous mapping causes an explosion in data volume. Storing millions of points—each appended with a high-dimensional floating-point descriptor like SIFT or SuperPoint—creates a massive memory bottleneck.
Simon Lynen et al. demonstrate in Get Out of My Lab: Large-scale, Real-Time Visual-Inertial Localization that city-scale maps easily exceed the RAM limits of mobile and embedded edge devices. To make localization truly pervasive across autonomous vehicles, drones, and mobile augmented reality, researchers are developing methods to compress or entirely redefine what a "map" is.
Compressing the Descriptor Space
If a system retains the underlying 3D point cloud structure, the primary culprit for storage bloat is the high-dimensional descriptor vector stored at every point. Torsten Sattler et al. address this in Hyperpoints and Fine Vocabularies for Large-Scale Location Recognition by replacing bulky raw descriptors with quantized visual vocabularies and co-visibility voting schemes, drastically lowering the memory footprint.
To push compression further, recent techniques optimize the descriptor representation directly through learned dimensionality reduction and quantization. Qiang Wang 0061 shows in MAD-DR: Map Compression for Visual Localization with Matchness Aware Descriptor Dimension Reduction that an asymmetric encoder-decoder network can compress 256-dimensional descriptors down to just 4 dimensions while maintaining matching accuracy.
[High-Dimensional Map] ──(Map Pruning / Filtering)──> [Salient Points Only]
│
(Product Quantization / Auto-encoder)
▼
[Compressed Low-Bit Map]
Similarly, Zakaria Laskar et al. introduce Differentiable Product Quantization for Memory Efficient Camera Relocalization, which pairs map pruning with a scene-specific "dequantization" network to fit maps into budgets as small as 1MB. By binarizing features or filtering out redundant points, as explored by Wentao Cheng et al. in Cascaded Parallel Filtering for Memory-Efficient Image-Based Localization, systems can discard the vast majority of non-essential visual data before it ever hits disk storage.
Bypassing Textures with Pure Geometry
Why store visual descriptors at all? If the geometric structure of the world is distinct enough, we can localize by matching shapes instead of pixels. Qunjie Zhou et al. demonstrate this intuition in Is Geometry Enough for Matching in Visual Localization?. Their framework, GoMatch, bypasses visual descriptors entirely, matching 2D keypoints to 3D maps using only bearing vectors and attention mechanisms, slashing storage requirements by over 98%.
This geometric abstraction is especially powerful when mapping structural lines rather than arbitrary points. Junho Kim et al. show in Fully Geometric Panoramic Localization that 2D and 3D lines provide a highly compact, privacy-preserving representation that is robust to extreme lighting changes. For aerial applications, Juelin Zhu et al. leverage this in LoD-Loc: Aerial Visual Localization using LoD 3D Map with Neural Wireframe Alignment, aligning a drone's camera view with lightweight 3D wireframe models instead of massive, multi-gigabyte textured meshes.
Semantic Abstraction and Object-Centric Mapping
Another strategy replaces millions of low-level points with a handful of high-level semantic objects. For autonomous driving, storing every point in a LiDAR scan or image sequence is prohibitively expensive. Wei-Chiu Ma demonstrates in Exploiting Sparse Semantic HD Maps for Self-Driving Vehicle Localization that using sparse semantic maps containing only lanes and traffic signs reduces storage by 99.7% while maintaining centimeter-level accuracy. Sheng Yang et al. follow a similar philosophy in Road Mapping and Localization using Sparse Semantic Visual Features, parameterizing standardized road elements to create maps roughly 5x smaller than conventional point maps. Kun Jiang also validates this in Monocular Vehicle Self-localization method based on Compact Semantic Map, showing that compact semantic maps bridge the gap between high-cost LiDAR and low-storage monocular systems.
Traditional Point Map: [ . . . . . . . . . . . . . . . . . . . . . . . . ] -> Millions of points
Semantic Abstracted: [ Lane Spline A ] ─────── [ Traffic Sign B ] -> Bytes per kilometer
This concept scales to 3D environments via scene graphs, object centroids, and discrete labels:
- Scene Graphs: Instead of saving raw point clouds, Francis Engelmann et al. present SceneGraphLoc: Cross-Modal Coarse Visual Localization on 3D Scene Graphs, which distills multi-modal scenes into fixed-size object embeddings, yielding a 1,000x reduction in storage.
- Object Centroids: For outdoor LiDAR mapping, Georgi Pramatarov et al. show in That's My Point: Compact Object-centric LiDAR Pose Estimation for Large-scale Outdoor Localisation that compressing a raw 1.41 MB scan down to a 1.33 kB collection of object centroids and semantic classes is sufficient for accurate cross-season localization.
- Segmentation Labels: The authors of SegLoc: Learning Segmentation-Based Representations for Privacy-Preserving Visual Localization replace high-dimensional vectors with clusters of discrete semantic labels, reducing storage requirements by orders of magnitude.
- 2D Map Priors: The authors of OrienterNet: Visual Localization in 2D Public Maps with Neural Matching match a neural Bird's-Eye View against standard 2D semantic maps (like OpenStreetMap), eliminating the need for explicit 3D maps entirely.
Compressing Range and Subsurface Data
The storage bottleneck applies equally to non-visual sensors like LiDAR and radar. Large-scale LiDAR mapping generates dense point clouds that consume significant bandwidth. To mitigate this, Yifan Duan et al. compress dense point clouds into elastic, lightweight spherical representations in CELLmap: Enhancing LiDAR SLAM through Elastic and Lightweight Spherical Map Representation.
Alternatively, Xinrui Wu et al. compress bulky LiDAR maps into 2D "Heat Maps" in LHMap-loc: Cross-Modal Monocular Localization Using LiDAR Point Cloud Heat Map, reducing map storage by 80% while running at 77 FPS. Ali Waqas utilizes lightweight HD-maps as 4DOF pose priors in HD-maps as Prior Information for Globally Consistent Mapping in GPS-denied Environments to bypass hundreds of gigabytes of survey-grade data. Even underground mapping faces this challenge; Teddy Ort et al. note in Autonomous Navigation in Inclement Weather Based on a Localizing Ground Penetrating Radar that reducing the complexity of localization maps is vital for scalable navigation.
Implicit Neural Maps and Coordinate Regression
The most radical shift away from explicit map storage is to compress the environment directly into the weights of a neural network. Instead of querying a database of points, the network learns to regress 3D coordinates directly from pixels. Shuzhe Wang et al. demonstrate this in HSCNet++: Hierarchical Scene Coordinate Classification and Regression for Visual Localization with Transformer, using a coarse-to-fine Transformer network to bypass explicit point cloud matching entirely.
Query Image ──> [ Neural Network / Transformer Weights ] ──> 3D Scene Coordinates
(The entire map is stored in the weights)
To balance the accuracy of feature matching with the compactness of neural networks, Shitao Tang et al. introduce NeuMap: Neural Coordinate Mapping by Auto-Transdecoder for Camera Localization. NeuMap encodes scenes into a grid of sparse latent codes and uses a scene-agnostic Transformer to decode 3D coordinates, achieving a 1000x storage reduction over traditional explicit maps.
Similarly, Qunjie Zhou et al. explore the compact, continuous representation of Neural Radiance Fields in The NeRFect Match: Exploring NeRF Features for Visual Localization, showing that implicit neural features can replace bulky external databases.
To make these neural approaches practical on consumer hardware, frameworks must handle multiple environments efficiently. Tristan Braud et al. show in MobileARLoc: On-device Robust Absolute Localisation for Pervasive Markerless Mobile AR that combining Absolute Pose Regressors (APRs) with Visual-Inertial Odometry (VIO) yields compact representations that support multiple environments without exhausting device memory. Finally, Tao Xie 0010 et al. introduce OFVL-MS: Once for Visual Localization across Multiple Indoor Scenes, which uses a single multi-task network with a layer-adaptive sharing policy to prevent linear storage scaling as the number of mapped environments grows.
Key Foundational Work
Get Out of My Lab: Large-scale, Real-Time Visual-Inertial Localization (2015) — contributed 1 concept elements to this direction.
Hyperpoints and Fine Vocabularies for Large-Scale Location Recognition (2015) — contributed 1 concept elements to this direction.
Maplab: An Open Framework for Research in Visual-Inertial Mapping and Localization (2017) — contributed 1 concept elements to this direction.