MARSIM: A Light-Weight Point-Realistic Simulator for LiDAR-Based UAVs

Fanze Kong, Xiyuan Liu, Benxu Tang, Jiarong Lin, Yunfan Ren, Yixi Cai, Fangcheng Zhu, Nan Chen, Fu Zhang

Simulating unmanned aerial vehicles (UAVs) is a high-stakes game. In the real world, a single coding error in a navigation algorithm can lead to a costly "rapid unscheduled disassembly." To avoid this, researchers rely on simulators. However, most existing tools struggle to replicate the messy, complex reality of the physical world because they rely on 3D mesh models—which are notoriously difficult to generate accurately from real-world scans.

Enter MARSIM, a light-weight, "point-realistic" simulator developed by researchers at the University of Hong Kong. Unlike its predecessors, MARSIM skips the mesh-making headache entirely, rendering realistic LiDAR scans directly from raw 3D point cloud data.

TL;DR

MARSIM is a high-performance, ROS-compatible simulator that generates realistic LiDAR scans directly from point cloud maps rather than 3D meshes. It is significantly faster and more memory-efficient than industry standards like Gazebo, allowing for high-fidelity UAV testing on standard consumer hardware.

What Problem Does This Paper Address?

Most mainstream simulators (like Gazebo or AirSim) require environments to be defined as 3D meshes. While meshes work well for "clean" man-made virtual worlds, they are a nightmare for representing the real world. Converting a real-world LiDAR scan of a forest or a complex building into a high-fidelity mesh is time-consuming and often results in "closed" surfaces where there should be gaps, or missing details due to occlusions.

Furthermore, rendering dense meshes is computationally expensive, often requiring high-end GPUs. This creates a bottleneck for researchers who need to run complex multi-UAV simulations or test algorithms on light-weight computing platforms (like an Intel NUC) typically found on actual drones.

Methods

The core innovation of MARSIM is its point-to-scan rendering pipeline. Instead of traditional ray-casting against a mesh, it uses the following steps:

  1. Preprocessing: The simulator imports a global point cloud map, performs spatial downsampling, and organizes points into voxels to speed up lookups.
  2. Depth Image Construction: It projects map points within the UAV's Field of View (FoV) onto a dense depth image.
  3. Occlusion Culling & Interpolation: To handle the sparsity of point clouds, MARSIM calculates an interpolation range ($\theta_{max}$) for each point. It uses a "smallest depth wins" strategy to ensure foreground objects correctly block background objects.
  4. Plane Correction: To prevent artifacts on slanted surfaces, the simulator treats map points as small local planes during interpolation, calculating the exact intersection of the "laser ray" with that plane.
  5. Pattern Masking: Finally, it masks the dense depth image with the specific scanning pattern of the target LiDAR (e.g., the non-repetitive flower-like pattern of a Livox Avia or the rings of a Velodyne).

The system is built on the Robot Operating System (ROS), making it "plug-and-play" for most existing robotics software stacks.

Datasets

The authors introduced a benchmark suite of 10 high-resolution (0.01m) real-world point cloud maps captured using hand-held LiDAR scanners. These include:

  • Natural Environments: Common, simple, and dense forests.
  • Indoor Spaces: Offices, machine workshops, and multi-layer corridors.
  • Infrastructure: Simple and complex parking garages, and the historic HKU main building.

These maps are specifically designed to test UAVs in scenarios with cluttered obstacles, narrow passages, and thin structures.

Key Insights and Ideas

  • Direct Point Cloud Utilization: By bypassing mesh reconstruction, MARSIM preserves the "fine details" of the environment that are usually lost. If the scanner saw a gap between two leaves, the simulator sees it too.
  • Efficiency through Projection: Projecting points onto a depth image is fundamentally more efficient for LiDAR simulation than casting millions of individual rays into a complex 3D scene.
  • Hardware Agnostic: While it supports GPU acceleration via OpenGL, the CPU-only version of MARSIM is still twice as fast as the GPU-accelerated version of Gazebo.
  • Closing the Sim-to-Real Gap: By using the exact same point cloud map for simulation that a drone would generate in the real world, the researchers showed that simulated flight trajectories almost perfectly match real-world experimental data.

Key Results

  • Speed: In high-resolution scenes (0.05m resolution), MARSIM’s GPU version is 10x faster than Gazebo.
  • Memory: MARSIM consumes roughly half the RAM of Gazebo for the same environments.
  • Real-time Performance: The simulator consistently maintains a rendering rate of over 10 Hz on an Intel NUC (i7-10710U), even with maps containing tens of millions of points.
  • Versatility: It successfully simulated various sensors (Livox Mid-360, Velodyne VLP-32, Intel RealSense D455) and supported multi-UAV swarm coordination and dynamic moving obstacles.

Limitations and Future Work

  • Map Quality Dependence: The realism of the simulation is strictly tied to the quality of the source point cloud. If the original scan has "ghosting" or significant noise, those errors will persist in the simulation.
  • Sparse Pattern Inefficiency: The authors note that for very sparse LiDARs (like a 16-beam spinning sensor), generating a full dense depth image first is somewhat wasteful. Future iterations might optimize the rendering pipeline specifically for low-resolution scanning patterns.
  • Dynamic Objects: While it supports moving spherical obstacles, simulating complex dynamic agents (like walking humans or other non-rigid bodies) remains a future challenge.

Summary updated

Abstract

The emergence of LiDAR sensors have brought new opportunities for autonomous unmanned aerial vehicles (UAVs) by advancing navigation safety and computation efficiency. Yet the successful developments of LiDAR-based UAVs must rely on extensive simulations. Existing simulators can hardly perform simulations of real-world environments due to the requirements of dense mesh maps that are difficult to obtain. Therefore, we develop a point-realistic simulator of real-world scenes for LiDAR-based UAVs. The key idea is the underlying point rendering method, where we construct a depth image directly from the point cloud map and interpolate it to obtain realistic LiDAR point measurements. Our developed simulator is able to run on a light-weight computing platform and supports the simulation of LiDARs with different resolution and scanning patterns, dynamic obstacles, and multi-UAV systems. Developed in the ROS framework, the simulator can easily communicate with other key modules of an autonomous robot, such as perception, state estimation, planning, and control. Finally, the simulator provides 10 high-resolution point cloud maps of various real-world environments, including forests of different densities, historic building, office, parking garage, and various complex indoor environments. Evaluation results show that the developed simulator achieves superior performance in terms of time and memory consumption against Gazebo and that the simulated UAV flights highly match the actual one in real-world environments. We believe such a point-realistic and light-weight simulator is crucial to bridge the gap between UAV simulation and experiments and will significantly facilitate the research of LiDAR-based autonomous UAVs in the future.
BibTeX
@article{kong2023marsim,
  author = {Fanze Kong and Xiyuan Liu and Benxu Tang and Jiarong Lin and Yunfan Ren and Yixi Cai and Fangcheng Zhu and Nan Chen and Fu Zhang},
  title = {{MARSIM: A Light-Weight Point-Realistic Simulator for LiDAR-Based UAVs}},
  journal = {IEEE Robotics Autom. Lett.},
  year = {2023},
  url = {https://openreview.net/forum?id=C4lzzhtbmH},
}
Figure