Toronto-3D: A Large-scale Mobile LiDAR Dataset for Semantic Segmentation of Urban Roadways

Weikai Tan, Nannan Qin, Lingfei Ma, Ying Li, Jing Du, Guorong Cai, Ke Yang, Jonathan Li

Abstract

Semantic segmentation of large-scale outdoor point clouds is essential for urban scene understanding in various applications, especially autonomous driving and urban high-definition (HD) mapping. With rapid developments of mobile laser scanning (MLS) systems, massive point clouds are available for scene understanding, but publicly accessible large-scale labeled datasets, which are essential for developing learning-based methods, are still limited. This paper introduces Toronto-3D, a large-scale urban outdoor point cloud dataset acquired by a MLS system in Toronto, Canada for semantic segmentation. This dataset covers approximately 1 km of point clouds and consists of about 78.3 million points with 8 labeled object classes. Baseline experiments for semantic segmentation were conducted and the results confirmed the capability of this dataset to train deep learning models effectively. Toronto-3D is released to encourage new research, and the labels will be improved and updated with feedback from the research community.

Introduction

Deep learning has revolutionized 3D scene understanding, yet these models remain highly data-dependent. Existing datasets for semantic segmentation—the task of labeling every point in a 3D scan—often fall short by being too small, having low point density, or lacking the complexity of real-world urban environments.

Furthermore, many datasets "clean" their data by trimming points far from the sensor. In reality, autonomous vehicles and mapping systems must deal with "raw" data where point density drops off significantly with distance or spikes when a vehicle stops at a red light. Toronto-3D addresses these challenges by providing a high-density, "untrimmed" dataset that reflects these realistic conditions.

Visualization of the Toronto-3D dataset: raw point cloud (top) and ground truth semantic labels (bottom).

The Toronto-3D Dataset

The Toronto-3D dataset was collected using a Teledyne Optech Maverick mobile laser scanning (MLS) system, which integrates a 32-line LiDAR, a panoramic camera, and high-precision GNSS/SLAM for positioning.

Dataset Characteristics

  • Scale: Approximately 1km of roadway, divided into four distinct sections.
  • Density: About 78.3 million points, averaging 1000 points/m² on the ground.
  • Classes: 8 labeled categories (Road, Road marking, Natural, Building, Utility line, Pole, Car, Fence) plus an "unclassified" tag.
  • Attributes: Each point includes XYZ coordinates, RGB color, Intensity, GPS time, Scan angle rank, and Label.
Top-down view of the full 1km roadway segmentation in Toronto-3D.

Toronto-3D offers a unique balance of high density and specific urban classes like "utility lines" and "road markings," which are vital for HD mapping but difficult to segment due to their thin or flush geometry.

Methodology: MS-TGNet

Standard architectures often struggle with the massive density fluctuations in raw MLS data. In Toronto-3D, point density can be 10x higher at intersections than on open stretches. To solve this, the authors proposed MS-TGNet (Multi-Scale Taylor Gaussian Network).

This model incorporates Multi-Scale Grouping (MSG) into graph convolution layers, allowing the network to capture geometric features at three different radii (0.2m, 0.4m, and 0.8m) simultaneously. This approach enhances the model's robustness to varying sampling rates typical of real-world mobile scanning.

Experimental Results

The researchers tested five state-of-the-art models—PointNet++, DGCNN, KPFCNN, MS-PCNN, and TGNet—against the proposed MS-TGNet.

  • Top Performer: MS-TGNet achieved the highest mean Intersection over Union (mIoU) of 60.96%, outperforming the baseline models.
  • Accuracy: KPFCNN achieved the highest Overall Accuracy (OA) at 91.71%, though it struggled more with minority classes compared to MS-TGNet.
  • Segmentation Challenges: All models performed poorly on "Road markings" and "Fences," with IoUs often falling below 22%. However, MS-TGNet’s multi-scale grouping was significantly better at identifying road markings (18.78% IoU) compared to several baselines that scored 0% on that class.
  • Contextual Errors: Models often misclassified truck tops as "buildings" or "fences" because the sensor only captures one side of the vehicle, making its 3D shape resemble a wall.

Conclusion

Toronto-3D provides a challenging benchmark for urban point cloud segmentation. While MS-TGNet demonstrated superior performance on difficult classes, hardware memory constraints currently limit the PointNet++ backbone's ability to process large-scale structures. Future work will focus on testing memory-efficient architectures like RandLA-Net and improving the classification of vertical structures prone to confusion.

Summary updated

Abstract

Semantic segmentation of large-scale outdoor point clouds is essential for urban scene understanding in various applications, especially autonomous driving and urban high-definition (HD) mapping. With rapid developments of mobile laser scanning (MLS) systems, massive point clouds are available for scene understanding, but publicly accessible large-scale labeled datasets, which are essential for developing learning-based methods, are still limited. This paper introduces Toronto-3D, a large-scale urban outdoor point cloud dataset acquired by a MLS system in Toronto, Canada for semantic segmentation. This dataset covers approximately 1 km of point clouds and consists of about 78.3 million points with 8 labeled object classes. Baseline experiments for semantic segmentation were conducted and the results confirmed the capability of this dataset to train deep learning models effectively. Toronto-3D is released to encourage new research, and the labels will be improved and updated with feedback from the research community.
BibTeX
@inproceedings{tan2020toronto,
  author = {Weikai Tan and Nannan Qin and Lingfei Ma and Ying Li and Jing Du and Guorong Cai and Ke Yang and Jonathan Li},
  title = {{Toronto-3D: A Large-scale Mobile LiDAR Dataset for Semantic Segmentation of Urban Roadways}},
  booktitle = {Workshops @ CVPR},
  year = {2020},
  url = {https://openreview.net/forum?id=scgh2LZYEZ},
}
Figure