Abstract
Semantic segmentation of 3D point cloud sequences is critical for autonomous navigation, yet existing methods often struggle to balance temporal consistency with computational efficiency. This paper presents TemporalLatticeNet, a framework that leverages the permutohedral lattice for efficient sparse convolutions. The core innovation is the Abstract Flow (AFlow) module, which matches features across time steps to distinguish between moving and stationary objects. Evaluated on the SemanticKITTI benchmark, the proposed model achieves a mean Intersection-over-Union (mIoU) of 47.1 while maintaining a high inference speed of 154ms per frame, significantly outperforming dense volumetric and 2D projection-based baselines.
1. Introduction
Assigning semantic labels—such as "car," "road," or "pedestrian"—to every point in a 3D LiDAR scan is a fundamental task for autonomous robots. While single-frame models have made significant progress, they possess a fundamental limitation: they lack temporal context. A single snapshot cannot distinguish between a parked car and a car temporarily stopped at a red light.
Existing temporal models often face a trade-off between accuracy and efficiency. Projecting 3D data into 2D images leads to a loss of depth information, while dense 3D grids are computationally prohibitive for real-time applications. This work addresses these challenges by processing raw 3D point cloud sequences directly and efficiently using a sparse lattice structure.
2. Methodology
2.1 The Permutohedral Lattice
The foundation of this work is LatticeNet, which maps sparse 3D points onto a permutohedral lattice. Unlike dense grids, this structure allows for fast convolutions on sparse data by only performing computations where points actually exist. This significantly reduces the memory footprint and increases processing speed.
2.2 TemporalLatticeNet and Recurrent Units
To incorporate temporal information, the researchers extended LatticeNet into TemporalLatticeNet. This architecture integrates recurrent units—specifically Long Short-Term Memory (LSTM) or Gated Recurrent Units (GRU)—at various stages of the network: Early, Middle, Bottleneck, and Late fusion. These units allow the model to pass a "memory" of previous timesteps to the current frame, facilitating the recognition of temporal patterns.
2.3 Abstract Flow (AFlow)
The primary contribution of this research is the Abstract Flow (AFlow) module. Inspired by the concept of optical flow in computer vision, AFlow enables the network to "track" how features move through the scene. Instead of simply comparing identical XYZ coordinates across frames, AFlow matches the features of a vertex in the current lattice with its "one-hop" neighbors in the previous timestep's lattice. By calculating similarity weights based on the distance between these abstract features, the model can account for object motion even when the underlying point distribution changes.
2.4 Coordinate Alignment
To ensure temporal consistency, the model utilizes a consistent lattice hash map across sequences. This ensures that 3D points at the same physical location across different timesteps map to the same coordinate vectors, simplifying the comparison of features over time.
3. Experiments and Results
3.1 Dataset and Setup
The model was evaluated using the SemanticKITTI dataset, a large-scale benchmark for LiDAR sequences in urban environments. SemanticKITTI is particularly suited for this task as it provides distinct labels for moving versus stationary objects (e.g., "moving-car" vs. "car").
3.2 Performance and Accuracy
The best-performing configuration (utilizing a GRU-GRU-AFlow-GRU setup) achieved a mean Intersection-over-Union (mIoU) of 47.1. This represents a significant improvement over established baselines such as SpSequenceNet (43.1 mIoU). Qualitative analysis confirms that the model is notably better at identifying moving vehicles in dense urban traffic where single-frame models typically fail.
3.3 Computational Efficiency
Efficiency is a key advantage of TemporalLatticeNet. Because the model is recurrent, it does not need to re-process the entire history of point clouds for every new frame; it simply updates its hidden state. The model processes a single frame in approximately 154ms, making it considerably faster than competitors like KPConv (225ms) and SpSequenceNet (477ms).
4. Discussion and Conclusion
4.1 Key Insights
A critical finding was the impact of reflectance values. Including the intensity of the laser return significantly improved performance, as it provides a unique "fingerprint" that assists the AFlow module in matching objects across frames. Furthermore, the results demonstrate that feature-based matching (AFlow) is superior to simple coordinate-based matching for dynamic scene understanding.
4.2 Limitations and Future Work
Despite its strengths, the model faces challenges with long-term temporal dependencies. If an object remains stationary for a duration exceeding the model's temporal window, it may be misclassified. Additionally, while efficient, the AFlow module does introduce a slight increase in memory consumption compared to simpler recurrent models.
Future research will focus on refining the directionality of the flow within the lattice space to provide a more granular understanding of movement, potentially enabling the model to predict future object trajectories directly within the segmentation pipeline.
Summary updated