Efficient representation of 3D spatial data for defense-related applications

Benjamin Kahl, Marcus Hebel, Michael ArensarXiv 2025 · DOI · arXiv

Introduction

In modern defense and security operations, traditional 2D maps are increasingly insufficient for complex tactical requirements. Commanders now require high-fidelity 3D "digital twins" of the battlespace to conduct critical tasks such as line-of-sight analysis, precise route planning, and immersive simulations. However, creating these models involves a significant technical trade-off between geometric accuracy and visual realism.

The 3D Representation Gap

Current 3D mapping technologies generally fall into two categories. Traditional representations, such as triangle meshes and point clouds, provide the geometric "truth" necessary for physics-based calculations. Yet, these often lack the photorealistic detail required for human situational awareness. Conversely, modern AI-driven techniques like Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting (3DGS) offer stunning visual quality but are frequently "geometrically hollow," containing artifacts that interfere with precise spatial analysis.

Alignment of 2D aerial imagery with 3D reconstructed point clouds.

Proposed Hybrid Architecture

To bridge this gap, this research proposes a hybrid architecture that combines the strengths of both paradigms. The system utilizes a traditional mesh scaffold to provide "geometric grounding" for a high-fidelity neural visual layer. The study primarily utilizes data from the MODISSA platform, a mobile sensor laboratory developed by Fraunhofer IOSB equipped with Lidar and 360° cameras.

The workflow follows a structured pipeline:

  1. Collection: Gathering multi-modal data from Lidar, cameras, IMU, and GPS.
  2. Fusion: Aligning raw sensor data into a coherent spatial frame.
  3. Aggregation: Building a hierarchical model using Bounding Volume Hierarchies (BVH) to manage large-scale scenes.
  4. Usage: Delivering the final aggregate model to user applications.
The end-to-end pipeline from sensor collection to the final aggregate model.

This architecture employs a dual-objective loss function during training. A Geometric Loss ensures the model adheres to raw Lidar data for spatial accuracy, while a Photometric Loss ensures the model matches camera imagery for visual fidelity.

Comparative Performance

Qualitative analysis across various metrics highlights the trade-offs involved. While 3DGS and Neural Fields dominate in visual fidelity, they lag in surface fidelity compared to point clouds and meshes. Triangle meshes remain the most efficient for downstream tasks like visibility and physics due to extensive GPU hardware optimization, whereas neural representations offer superior memory efficiency for scene compression.

Key Insights: The Necessity of Grounding

A critical finding of this study is the "hollow" nature of pure neural reconstructions. Because NeRFs and 3DGS are optimized primarily to reconstruct 2D images rather than 3D volumes, they often produce "floaters"—transparent artifacts that look correct from a camera's perspective but break line-of-sight algorithms.

The authors argue that "Geometric Grounding"—using an explicit mesh as the "bones" for the neural "skin"—is mandatory for tactical applications. Furthermore, scalability is achieved through hierarchy; by dividing scenes into manageable instances using BVH, the system can implement "Level of Detail" (LOD) management, prioritizing computational resources for nearby objects.

Conclusion and Future Work

While the proposed framework offers a path toward the "best of both worlds" in 3D mapping, several challenges remain. Future research must address the complexities of dual-optimization to prevent mesh and neural layers from conflicting, improve "write performance" for real-time mission updates, and extend the model to handle dynamic, changing environments.

Summary updated

Abstract

Geospatial sensor data is essential for modern defense and security, offering indispensable 3D information for situational awareness. This data, gathered from sources like lidar sensors and optical cameras, allows for the creation of detailed models of operational environments. In this paper, we provide a comparative analysis of traditional representation methods, such as point clouds, voxel grids, and triangle meshes, alongside modern neural and implicit techniques like Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting (3DGS). Our evaluation reveals a fundamental trade-off: traditional models offer robust geometric accuracy ideal for functional tasks like line-of-sight analysis and physics simulations, while modern methods excel at producing high-fidelity, photorealistic visuals but often lack geometric reliability. Based on these findings, we conclude that a hybrid approach is the most promising path forward. We propose a system architecture that combines a traditional mesh scaffold for geometric integrity with a neural representation like 3DGS for visual detail, managed within a hierarchical scene structure to ensure scalability and performance.
BibTeX
@misc{kahl2025efficient,
  author = {Benjamin Kahl and Marcus Hebel and Michael Arens},
  title = {{Efficient representation of 3D spatial data for defense-related applications}},
  year = {2025},
  eprint = {2511.05109},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.05109},
}
Figure