Toronto-3D: A Large-scale Mobile LiDAR Dataset for Semantic Segmentation of Urban Roadways

Weikai Tan, Nannan Qin, Lingfei Ma, Ying Li, Jing Du, Guorong Cai, Ke Yang, Jonathan Li

Abstract

Semantic segmentation of large-scale outdoor point clouds is essential for urban scene understanding in various applications, especially autonomous driving and urban high-definition (HD) mapping. With rapid developments of mobile laser scanning (MLS) systems, massive point clouds are available for scene understanding, but publicly accessible large-scale labeled datasets, which are essential for developing learning-based methods, are still limited. This paper introduces Toronto-3D, a large-scale urban outdoor point cloud dataset acquired by a MLS system in Toronto, Canada for semantic segmentation. This dataset covers approximately 1 km of point clouds and consists of about 78.3 million points with 8 labeled object classes. Baseline experiments for semantic segmentation were conducted and the results confirmed the capability of this dataset to train deep learning models effectively. Toronto-3D is released to encourage new research, and the labels will be improved and updated with feedback from the research community.

Introduction

For autonomous vehicles to navigate safely, they must perform accurate semantic segmentation: the process of classifying millions of individual laser-reflected points into meaningful categories like pedestrians, vehicles, or road infrastructure. While deep learning has revolutionized this field, these models are notoriously data-hungry.

Existing outdoor LiDAR datasets often suffer from low point density or a lack of specific labels—such as lane arrows or utility lines—that are critical for complex urban navigation. Furthermore, many datasets are "pre-cleaned," removing the variable point densities that sensors encounter in real-world scenarios. Toronto-3D was created to provide a more realistic, large-scale, and diverse training ground for the next generation of urban scene understanding models.

The Toronto-3D Dataset

The primary contribution of this work is the Toronto-3D dataset, which comprises 78.3 million points captured along a 1 km stretch of Avenue Road in Toronto, Canada. The data was acquired using a vehicle-mounted Teledyne Optech Maverick system, integrating a 32-line LiDAR sensor with panoramic cameras and high-precision GPS/SLAM positioning.

Overview of the Toronto-3D dataset showing the four sections (L001-L004) and the corresponding labeled point cloud.

Toronto-3D preserves the "messiness" of real-world scanning. For instance, when a sensor-equipped car stops at a red light, it captures redundant scans of the immediate vicinity, while distant objects are captured with lower density. This variability tests the robustness of segmentation models. The dataset includes eight semantic classes: road, road marking, natural, building, utility line, car, fence, and unclassified. Notably, separating "road markings" from the general "road" category is essential for autonomous systems to understand the "language" of the roadway.

Methodology and Baseline Testing

Every point in the dataset was manually classified using CloudCompare software. To evaluate the dataset's utility, the researchers tested several state-of-the-art point cloud models, including PointNet++, DGCNN, and KPFCNN.

Detail of a point cloud segment and its semantic classification into eight distinct urban classes.

In addition to these baselines, the authors introduced MS-TGNet (Multi-Scale Taylor Gaussian Network). This architecture incorporates a Multi-Scale Grouping layer that allows the model to analyze the point cloud at different neighborhood sizes (0.2m, 0.4m, and 0.8m) simultaneously. This multi-scale approach is specifically designed to handle the fluctuating point densities inherent in mobile laser scanning.

Results and Discussion

The baseline experiments highlighted both the strengths and current limitations of semantic segmentation models. While models are generally effective at identifying large structures like buildings, they struggle with fine-grained urban features:

  • Overall Performance: KPFCNN achieved the highest overall accuracy of 91.71%.
  • mIoU Leader: The proposed MS-TGNet achieved the best mean Intersection over Union (mIoU) of 60.96%, validating the importance of multi-scale analysis.
  • The Challenge of Detail: All models struggled with "hard" classes like Road Markings and Fences, where IoU scores often fell below 22%.
  • MS-TGNet Advantage: MS-TGNet was particularly effective at identifying road markings (18.78% IoU), whereas KPFCNN failed to identify them entirely (0% IoU) in the baseline tests.

Conclusion

Toronto-3D provides a high-density, realistic benchmark for urban semantic segmentation. The results demonstrate that while overall accuracy is high, significant work remains to improve the detection of thin or low-profile objects like utility lines and road markings. Future work will explore memory-efficient architectures, such as RandLA-Net, to process larger segments of the dataset simultaneously. Toronto-3D is intended to be a living dataset that will evolve with community feedback to support the advancement of autonomous driving and HD mapping technologies.

Summary updated

Abstract

Semantic segmentation of large-scale outdoor point clouds is essential for urban scene understanding in various applications, especially autonomous driving and urban high-definition (HD) mapping. With rapid developments of mobile laser scanning (MLS) systems, massive point clouds are available for scene understanding, but publicly accessible large-scale labeled datasets, which are essential for developing learning-based methods, are still limited. This paper introduces Toronto-3D, a large-scale urban outdoor point cloud dataset acquired by a MLS system in Toronto, Canada for semantic segmentation. This dataset covers approximately 1 km of point clouds and consists of about 78.3 million points with 8 labeled object classes. Baseline experiments for semantic segmentation were conducted and the results confirmed the capability of this dataset to train deep learning models effectively. Toronto-3D is released to encourage new research, and the labels will be improved and updated with feedback from the research community.
BibTeX
@inproceedings{tan2020toronto,
  author = {Weikai Tan and Nannan Qin and Lingfei Ma and Ying Li and Jing Du and Guorong Cai and Ke Yang and Jonathan Li},
  title = {{Toronto-3D: A Large-scale Mobile LiDAR Dataset for Semantic Segmentation of Urban Roadways}},
  booktitle = {Workshops @ CVPR},
  year = {2020},
  url = {https://openreview.net/forum?id=scgh2LZYEZ},
}
Figure