TL;DR
MuSCLE is a novel compression framework that slashes the storage requirements of LiDAR data streams by exploiting the "memory" of the sensor. By using deep entropy models to capture spatio-temporal redundancies across successive LiDAR sweeps, it achieves a 7–35% bitrate reduction over previous state-of-the-art methods while maintaining high accuracy for downstream tasks like object detection and semantic segmentation.
What Problem Does This Paper Address?
LiDAR sensors are the "eyes" of autonomous vehicles, but they are incredibly chatty. A single 64-line sensor can generate over 3 billion points per hour. Storing and transmitting this massive influx of 3D data is a logistical nightmare for engineers.
While image and video compression are mature fields, point cloud compression is much harder because the data is sparse, unstructured, and 3D. Most existing solutions treat each LiDAR "sweep" (a 360-degree rotation) as an independent event, ignoring the fact that the world doesn't change much in the 100 milliseconds between frames. Furthermore, most methods focus only on the geometry (the $x, y, z$ coordinates) and neglect the "intensity" (the reflectivity of the surface), which is vital for many AI perception models.
Methods
The authors propose MuSCLE (Multi Sweep Compression of LiDAR using Entropy models) to address these inefficiencies. The pipeline works in several key stages to jointly compress geometry and intensity:
- Octree Representation: The 3D space is partitioned into an octree—a tree structure where each node represents a cube of space. If a cube contains points, it's subdivided into eight smaller "octants." This continues until a desired precision is reached.
- Conditional Entropy Modeling: This is the core innovation. Instead of using a static probability model to encode the octree, MuSCLE uses a deep neural network to predict the probability of a node being occupied based on two contexts:
- Ancestral context: Information from higher levels of the current tree (top-down pass).
- Temporal context: Information from the previous LiDAR sweep.
- Continuous Convolutions: Because points in two different sweeps never align perfectly, the model uses "continuous convolutions" to aggregate features from the previous frame. This allows the model to "look" at the neighborhood of a point in the past to predict its state in the present without requiring strict voxel grids.
- Intensity Compression: The same spatio-temporal logic is applied to intensity values. By predicting the intensity of a point based on its neighbors in the previous frame, the model can encode the actual values using far fewer bits.
- Lossless Coding: The predicted probabilities are fed into an arithmetic range coder to produce the final, highly compressed bitstream.
Datasets
The researchers evaluated MuSCLE on two massive, industry-standard datasets:
- UrbanCity: A large-scale dataset collected by self-driving vehicles across North American cities, featuring 5,000 sequences.
- SemanticKITTI: A popular benchmark for LiDAR point cloud sequences that includes dense semantic labels for every point.
Key Insights and Ideas
- Temporal Redundancy is King: By simply looking at the previous frame, the researchers reduced the spatial bitrate by up to 45% on SemanticKITTI. This confirms that the "static" parts of the world (roads, buildings) provide a massive opportunity for compression.
- Continuous Convolutions as an Aligner: Traditional convolutions require a grid. By using continuous convolutions, MuSCLE can relate unstructured 3D points across time without needing explicit "tracking" or point-to-point correspondences.
- Preserving "Realism": The paper argues that bitrate isn't the only metric that matters. They show that AI models (like segmentors) perform better on MuSCLE-compressed data than on data compressed by other methods at the same bitrate. This suggests MuSCLE preserves the features that actually matter for machine "vision."
Key Results
MuSCLE demonstrates state-of-the-art performance across multiple metrics, proving highly effective at reducing bitrates while preserving geometric fidelity.
- Bitrate Reduction: MuSCLE achieved a 7–17% reduction on UrbanCity and a 15–35% reduction (joint geometry and intensity) on SemanticKITTI compared to OctSqueeze, the previous state-of-the-art deep learning method.
- Superiority over Standards: It significantly outperformed the MPEG TMC13 standard, especially at lower bitrates where MPEG tends to produce more artifacts. This is evident in the rate-distortion curves where MuSCLE maintains higher F1 scores and PSNR at lower bits-per-point.
- Downstream Performance: When testing on object detection (AP) and semantic segmentation (mIoU), MuSCLE-compressed data showed the smallest "realism gap," meaning the AI's performance stayed closest to what it would be on uncompressed "Oracle" data.
Limitations and Future Work
While MuSCLE is a significant step forward, the authors note a few areas for growth:
- Sensor Suites: Currently, the model only handles LiDAR. The authors suggest that future work could involve "cross-modal" compression—using camera images to help compress LiDAR data or vice versa.
- Computational Complexity: The use of continuous convolutions and deep entropy models is computationally more intensive than simple geometric codecs. While the paper mentions parallelization, real-time deployment on edge hardware remains a challenge for deep learning-based compression.
- Sequential Dependency: Because the model relies on the previous sweep, a single corrupted frame could theoretically impact the decoding of subsequent frames (a common issue in video compression known as error propagation).
Summary updated