Robust Monocular Localization of Drones by Adapting Domain Maps to Depth Prediction Inaccuracies

Priyesh Shukla, Sureshkumar S., Alex C. Stutts, Sathya Ravi, Theja Tulabandhula, Amit Ranjan Trivedi

TL;DR

Researchers have developed a "cross-modal" framework for drone localization that combines lightweight deep learning with traditional Bayesian filtering. By mathematically "warping" a 3D map to compensate for the systematic errors of a tiny depth-prediction model, they achieved high-precision navigation that is robust to extreme lighting and requires significantly less computational power than standard end-to-end neural networks.

What Problem Does This Paper Address?

For a drone to navigate autonomously, it needs to know its "pose"—its exact position and orientation in space. While heavy sensors like LiDAR or stereo cameras make this easy, they are power-hungry and bulky. Monocular (single-camera) localization is the "holy grail" for tiny drones, but it’s notoriously difficult because a single image lacks absolute depth information.

Current solutions usually fall into two camps:

  1. Traditional Filtering: Robust and interpretable, but often requires expensive sensors to resolve distance ambiguities.
  2. End-to-End Deep Learning (e.g., PoseNet): Great at pattern recognition but acts as a "black box." These models are brittle; if you move a chair or change the lighting, the whole network often needs to be retrained. They also struggle to scale to new environments without massive new datasets.

The authors address this gap by asking: Can we use a very small, "imperfect" neural network for depth and fix its mistakes using a smart geometric filter?

Methods

The authors propose a split architecture:

  • The Front-End (Depth Prediction): They use a lightweight Convolutional Neural Network (CNN) based on MobileNet-v2 and RefineNet. This model is intentionally kept small (down to 1.09M parameters) to run on edge hardware. Because it's small, its depth estimates are often "noisy" or systematically biased.
  • The Back-End (Pose Reasoning): Instead of a second neural network for pose, they use a Bayesian Particle Filter. This filter compares the predicted depth against a pre-existing 3D map of the environment.
  • The "Secret Sauce" (Map Adaptation): This is the paper's core innovation. Rather than trying to force the tiny depth network to be perfect (which would require a much larger model), they use a small Multi-Layer Perceptron (MLP) to adapt the map. The MLP learns to shift and rotate the 3D map points so they align with the specific inaccuracies of the depth predictor.

The map itself is represented as a Gaussian Mixture Model (GMM), which is far more memory-efficient than standard point clouds or voxel grids.

Datasets

The framework was evaluated using:

  • NYU-Depth V2: Used for the initial training of the depth prediction network. This allows the model to learn general "depth-sensing" capabilities from a large, unrelated dataset.
  • RGBD Dataset (Scene-02): Used to test the localization performance in specific indoor environments.
  • Extreme Lighting Models: The authors simulated three lighting conditions—High Brightness (L1), Medium Light (L2), and Very Dim Light (L3)—to test the limits of the vision system.

Key Insights and Ideas

  • Map-Independence vs. Map-Dependence: The authors argue that depth estimation is a general task (a wall looks like a wall anywhere), while pose estimation is environment-specific. By using AI only for the general task (depth) and traditional geometry for the specific task (pose), the system becomes much more adaptable.
  • Embracing Inaccuracy: Most researchers try to minimize depth error. This paper suggests that as long as the error is systematic, you can simply "warp" your world map to match the "vision" of your robot.
  • Resource Efficiency: By openly representing the map and intermediate features, the system allows for "reusable" data. The same depth map used for localization can be instantly plugged into an obstacle avoidance algorithm without extra computation.

Key Results

The results demonstrate a significant leap in robustness:

  • Accuracy vs. Model Size: Even when the depth network was shrunk by 66% (from 3.1M to 1.09M parameters), the pose accuracy remained stable thanks to the map adaptation.
  • Lighting Robustness: In "very dim" conditions where standard deep learning models (like PoseNet) failed significantly, this framework maintained high accuracy.
  • Quantitative Edge: In head-to-head tests, the proposed model achieved a Pose RMSE of 11.4 cm, compared to 58.9 cm for PoseNet and 26.7 cm for traditional GMM-based filtering alone.
  • Adaptability: The system achieved "Very Good" light variation tolerance, whereas PoseNet was rated "Low."

Limitations and Future Work

While the framework is efficient during inference (running on the drone), the joint training process is computationally expensive. It involves "nested sampling," where the system has to run particle filtering simulations during the learning phase.

The authors also note that while GMMs are memory-efficient, they still require a pre-scanned map of the environment. Future work could involve integrating SLAM (Simultaneous Localization and Mapping) to allow the drone to build these adaptive GMM maps on the fly in completely unknown environments.

Summary updated

BibTeX
@inproceedings{shukla2023robust,
  author = {Priyesh Shukla and Sureshkumar S. and Alex C. Stutts and Sathya Ravi and Theja Tulabandhula and Amit Ranjan Trivedi},
  title = {{Robust Monocular Localization of Drones by Adapting Domain Maps to Depth Prediction Inaccuracies}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2023},
  doi = {10.1109/icassp49357.2023.10096758},
}
Figure