Research on Distance Transform and Neural Network Lidar Information Sampling Classification-Based Semantic Segmentation of 2D Indoor Room Maps

Tao Zheng, Zhizhao Duan, Jin Wang, Guodong Lu, Shengjie Li, Zhiyong YuSensors 2021 · DOI

TL;DR

This paper presents a novel, efficient, and high-precision method for semantic segmentation of 2D indoor room maps using Lidar data. By combining distance transform watershed pre-segmentation with a custom lightweight neural network (LCNet) for classification, the approach achieves robust semantic labeling suitable for low-power mobile robots like those used for sweeping.

What Problem Does This Paper Address?

Mobile robots operating in indoor environments need to understand their surroundings to navigate and perform tasks effectively. This requires semantic segmentation of their maps, which means labeling different areas (e.g., rooms, corridors, doorways). Existing methods often struggle with computational complexity, making them unsuitable for resource-constrained robots like those with Raspberry Pi processors. Furthermore, accurately distinguishing between similar-looking areas, especially at junctions, remains a challenge. This paper aims to bridge this gap by developing a method that is both accurate and computationally efficient, enabling semantic understanding on low-power platforms.

Methods

The proposed method has two main stages: training and testing.

Training Stage: * Simulated Lidar Data: A simulated Lidar is used to generate "lidar detection line maps" for each point in a manually labeled 2D room map. These maps capture the spatial information around each point. * Lightweight Neural Network (LCNet): A custom neural network, LCNet, is designed. It's inspired by ResNet-18 but significantly optimized for low-power devices. Key modifications include removing the 7x7 convolution in the first layer, using a smaller input size (48x48), and replacing standard ResBlocks with a custom LCBlock that incorporates depthwise convolution and Octconv layers. This reduces parameters and computational load. * Training: LCNet is trained using the simulated lidar line maps and their corresponding semantic labels.

Testing Stage: * Pre-Segmentation: The input binary map is first pre-segmented into simpler cells using a combination of distance transform and watershed algorithms. This step helps to group similar areas and reduce the overall processing load. * Optimized Sampling: Instead of processing every point, the method identifies "optimized areas" within the pre-segmented cells. These areas are derived from the distance transform results and are designed to avoid noisy boundary regions. Sparse sampling points are then selected within these optimized areas. * Classification: The lidar detection line maps from these sparse, optimized sampling points are fed into the trained LCNet for classification. * "Winner-Take-All" Principle: For each pre-segmented cell, the semantic label that receives the highest proportion of votes from the sampled points is assigned to the entire cell. This principle helps to consolidate the classification results and filter out noise. * Doorway Extraction: A specific algorithm is used to identify doorways, which are often located at the dividing lines between rooms and corridors. This involves analyzing the dividing lines and using classification results to label them as doorways.

Datasets

The paper uses manually labeled 2D indoor room maps for training and testing. Specifically, four maps are used as the training dataset for LCNet, and the experiments are conducted on these and other publicly available maps. The authors also mention using a lab environment to collect real data for experiments.

Key Insights and Ideas

  • Bridging the Gap for Edge Devices: The core innovation is the development of a lightweight neural network (LCNet) that achieves competitive accuracy while being efficient enough to run on low-power embedded systems like the Raspberry Pi. This is crucial for real-world mobile robot applications.
  • Pre-segmentation for Efficiency: Using distance transform and watershed algorithms to pre-segment the map into simpler cells significantly reduces the number of areas that need detailed classification, improving speed.
  • Optimized Sparse Sampling: The idea of sampling only from "optimized areas" derived from distance transform results is key. This avoids processing every pixel and focuses on more informative regions, further boosting efficiency and robustness by mitigating the impact of noisy boundary points.
  • "Winner-Take-All" for Robustness: Applying this principle at the cell level aggregates classification results from sparse samples, making the final semantic labeling more robust to individual misclassifications.
  • Simulated Lidar Data for Training: Generating synthetic lidar data from labeled maps provides a rich dataset for training, overcoming the scarcity of real-world labeled data.

Key Results

  • Accuracy: The proposed method achieved a mean recall of 0.965 and a mean precision of 0.943.
  • Speed: The LCNet model is 3.37 times faster than ResNet-18 on a PC.
  • Efficiency: The method significantly reduces the number of points that need to be classified compared to pixel-by-pixel approaches.
  • Resource Constraints: The LCNet model is small enough (3.4M) to be deployed on a Raspberry Pi, whereas ResNet-18 (44.7M) is not feasible.
  • Comparison: The proposed method outperforms Voronoi and morphological segmentation methods in terms of recall and accuracy, and it uniquely provides semantic labels.

Limitations and Future Work

  • Limitations:
    • The paper notes that while LCNet is slightly less accurate than ResNet-18 on a PC (91.2% vs. 93.6%), its efficiency and deployability on edge devices are its main advantages.
  • The classification on Raspberry Pi can still be time-consuming when processing a large number of sampled points, although the sparse sampling helps.
  • Future Work:
    • The authors suggest that further improvements in running speed can be achieved by analyzing the spatial correlation of adjacent sampling points to further reduce the number of points needing classification.
  • The successful deployment on a prototype mobile robot demonstrates the feasibility and high efficiency of the method, suggesting its direct applicability to real-world robotic systems.

Summary updated

Abstract

Semantic segmentation of room maps is an essential issue in mobile robots' execution of tasks. In this work, a new approach to obtain the semantic labels of 2D lidar room maps by combining distance transform watershed-based pre-segmentation and a skillfully designed neural network lidar information sampling classification is proposed. In order to label the room maps with high efficiency, high precision and high speed, we have designed a low-power and high-performance method, which can be deployed on low computing power Raspberry Pi devices. In the training stage, a lidar is simulated to collect the lidar detection line maps of each point in the manually labelled map, and then we use these line maps and the corresponding labels to train the designed neural network. In the testing stage, the new map is first pre-segmented into simple cells with the distance transformation watershed method, then we classify the lidar detection line maps with the trained neural network. The optimized areas of sparse sampling points are proposed by using the result of distance transform generated in the pre-segmentation process to prevent the sampling points selected in the boundary regions from influencing the results of semantic labeling. A prototype mobile robot was developed to verify the proposed method, the feasibility, validity, robustness and high efficiency were verified by a series of tests. The proposed method achieved higher scores in its recall, precision. Specifically, the mean recall is 0.965, and mean precision is 0.943.
BibTeX
@inproceedings{zheng2021research,
  author = {Tao Zheng and Zhizhao Duan and Jin Wang and Guodong Lu and Shengjie Li and Zhiyong Yu},
  title = {{Research on Distance Transform and Neural Network Lidar Information Sampling Classification-Based Semantic Segmentation of 2D Indoor Room Maps}},
  booktitle = {Sensors},
  year = {2021},
  doi = {10.3390/s21041365},
}
Figure