Traditional Simultaneous Localization and Mapping (SLAM) systems rely on explicit geometric representations like point clouds, surfels, or voxel grids. While these structures scale predictably, they lack the ability to interpolate smoothly across unobserved regions or provide compact, photorealistic renderings. Modern neural implicit mapping—pioneered by frameworks like Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS)—solves this by encoding geometry and appearance within continuous functions.
However, as noted by Yang et al. in their survey SLAM Meets NeRF: A Survey of Implicit SLAM Methods, standard neural architectures are fundamentally ill-suited for the sequential nature of live robotics. When a robot explores an environment, it receives a continuous stream of highly correlated local observations. Optimizing a network on this streaming data violates the independent and identically distributed (i.i.d.) assumption required for stable neural network training, leading directly to catastrophic forgetting.
Why Neural Maps Forget: The Mechanics of Drift
To understand why neural maps degrade, consider a standard Multilayer Perceptron (MLP) mapping a multi-room house. The network's weights are shared globally; every parameter potentially influences the predicted geometry of every room.
When the robot enters Room B, the optimization process adjusts the network's weights to minimize reconstruction error for Room B's walls. Because these parameters are shared globally, this local update inadvertently distorts the representation of Room A, which the robot visited twenty minutes prior.
Monolithic MLP (Global Weights)
[Stream of Room B Data] ──> ∇ Optimization ──> Updates Global Weights ──> [Room A Distorted]
Long et al. describe this in Learning Barrier Functions with Memory for Robust Safe Navigation as a form of "representational drift." As the network adapts to new local geometry, its global consistency collapses.
This problem is not unique to coordinate MLPs; it also plagues explicit continuous representations like 3D Gaussian Splatting. As Sun et al. demonstrate in High-Fidelity SLAM Using Gaussian Splatting with Rendering-Guided Densification and Regularized Optimization, unconstrained optimization overfits the map's parameters to the current camera view, corrupting previously established geometry. Even recurrent architectures designed to compress history into a fixed-size state suffer from this issue, as Chen et al. show in TTT3R: 3D Reconstruction as Test-Time Training, where the model naturally prioritizes recent visual data over older observations, causing severe trajectory drift.
Strategy 1: Experience Replay and Keyframe Curation
The most intuitive remedy for catastrophic forgetting is to break the sequential correlation of the data stream by reintroducing past observations during the optimization loop. This is known as experience replay.
In the seminal neural SLAM work iMAP: Implicit Mapping and Positioning in Real-Time, Sucar et al. introduced a replay buffer of keyframes. By jointly optimizing the network on a mix of current frames and randomly sampled historical keyframes, they successfully balanced the stability-plasticity tradeoff.
Replay Buffer Approach
[Current Frame] ──┐
├──> Joint Optimization ──> Updates Global Weights
[Saved Keyframe] ─┘
However, storing every historical frame is memory-prohibitive for long-duration trajectories. To address this, Taner et al. proposed a "Similarity-based Subset" (SimS) curation strategy in Channel Charting for Streaming CSI Data. Instead of a simple sliding window—which discards old data completely—their approach maintains a sparse, high-quality "core" dataset that represents the global physical space, ensuring the network retains a stable global map without a ballooning memory footprint.
Strategy 2: Spatial Partitioning and Hybrid Representations
While experience replay mitigates forgetting, it treats the symptom rather than the cause. The root cause remains the global coupling of parameters. A parallel line of research eliminates this coupling by replacing monolithic networks with spatially localized, hybrid representations.
The core intuition is simple: if a parameter only influences a specific, bounded region of 3D space, optimizing it for Room B cannot physically alter Room A.
Hybrid Representation (Localized Weights)
[Stream of Room B Data] ──> ∇ Optimization ──> Updates Voxel B Embeddings ──> [Voxel A Untouched]
Yang et al. operationalized this in Vox-Fusion: Dense Tracking and Mapping with Voxel-based Neural Implicit Representation by utilizing a dynamically expanding sparse octree to store neural feature embeddings. Instead of querying a global MLP, the system looks up local embeddings within the active voxel and passes them to a tiny, shallow network.
Similarly, Johari introduced ESLAM: Efficient Dense SLAM System Based on Hybrid Representation of Signed Distance Fields, which projects features onto multi-scale planes. By localizing the optimization to specific coordinate planes, the system prevents updates from degrading distant parts of the trajectory.
Taking this further, Guo et al. developed NeuV-SLAM: Fast Neural Multiresolution Voxel Optimization for RGBD Dense SLAM, which combines a hash-based management system with multiresolution neural voxels. This spatial partitioning ensures that updates are strictly bounded, allowing the map to scale to larger environments without overwriting historical geometric information.
Strategy 3: Global Regularization and Bundle Adjustment
For explicit continuous representations like 3D Gaussian Splatting, spatial partitioning is inherent, yet forgetting still occurs because the optimization process itself can deform the global coordinate frame or misalign overlapping views. In these scenarios, researchers rely on explicit geometric regularization and global optimization techniques.
In GO-SLAM: Global Optimization for Consistent 3D Instant Reconstruction, Zhang addresses the drift caused by local updates by integrating online full bundle adjustment (BA) with a neural implicit surface representation. By continuously optimizing all camera poses and the global map structure simultaneously, loop closures can propagate corrections backward through the entire trajectory, resolving inconsistencies.
For large-scale multi-room environments, Shen proposed PLGSLAM: Progressive Neural Scene Representation with Local to Global Bundle Adjustment, which uses a progressive scene representation alongside local-to-global bundle adjustment. This ensures that while local details are refined rapidly, the global structural integrity of previously mapped rooms is preserved via a macro-level optimization graph. By decoupling local high-frequency updates from global low-frequency corrections, the system maintains long-term memory across expansive trajectories.
Key Foundational Work
Learning Barrier Functions with Memory for Robust Safe Navigation (2020) — contributed 1 concept elements to this direction.
iMAP: Implicit Mapping and Positioning in Real-Time (2021) — contributed 1 concept elements to this direction.