CrashCar101: Procedural Generation for Damage Assessment

Jens Parslov, Erik Riise, Dim P. PapadopoulosWACV 2024

TL;DR

This paper introduces CrashCar101, a novel synthetic dataset for vehicle damage and part segmentation, generated procedurally from 3D car models. The authors demonstrate that training models on CrashCar101 significantly improves performance on real-world datasets for both damage and part segmentation, especially in few-shot learning scenarios, highlighting the effectiveness of synthetic data for bridging the sim2real gap.

What Problem Does This Paper Address?

Automatically assessing vehicle damage is crucial for various applications, from insurance claims to accident response. This task requires precise per-pixel segmentation of both the damaged areas and the specific car parts that are affected. However, creating large, manually annotated datasets for this purpose is extremely expensive and time-consuming, often leading to errors. Existing datasets are either too small, lack detailed annotations, or focus on image-level classification rather than segmentation. This paper addresses the need for a large-scale, pixel-accurate dataset to train robust damage and part segmentation models for vehicles.

Methods

The core of this paper is a procedural generation pipeline for creating synthetic vehicle damage data. The process involves:

  • 3D Model Acquisition and Annotation: Utilizing 3D car models from ShapeNetCore, the authors manually annotate sub-meshes with 27 semantic part categories. To speed up this annotation process, they employ a human-in-the-loop approach using a DeepLabv3 model for initial predictions and then refining them.
  • Procedural Damage Generation: A pipeline is developed to simulate five common damage types: dents, scratches, cracks, glass shatter, and broken lamps. These damages are applied by manipulating the 3D mesh geometry and physically-based material properties of the car models. The generators use interpretable input parameters that can be sampled to create a wide variety of damage appearances and scales.
  • Scene Generation and Rendering: The damaged 3D car models are placed in realistic urban environments (using HDRI backgrounds) with randomized camera viewpoints and car colors. This process generates 2D images that are paired with perfect, pixel-accurate ground-truth segmentation masks for both car parts and damage types.
  • CrashCar101 Dataset Creation: The pipeline is executed to generate the CrashCar101 dataset, comprising 101,050 synthetic 2D images with corresponding part and damage annotations. A subset, CrashCar101-Part, containing only part annotations, is also created for specific experiments.

Datasets

  • Training Datasets:
    • Pascal-Part [8] (real, 490 images)
    • CGPART [32] (synthetic, 31,448 images)
    • CrashCar101-Part (synthetic, 14,175 images)
    • Combination of CrashCar101-Part + Pascal-Part
  • Evaluation Datasets:
    • Pascal-Part [8] (real, 62 images)
    • UDAPART [33] (real, 40 images)
    • CG-PART [32] (real, 349 images)
    • CarDD [63] (real, 349 images for damage segmentation)

The paper introduces the CrashCar101 dataset as a novel, large-scale synthetic benchmark for vehicle damage and part segmentation. It is distinguished by its procedural generation, providing pixel-perfect annotations and high variability, unlike existing real-world datasets which are often limited in size and annotation quality.

Key Insights and Ideas

  • Synthetic Data for Sim2Real Transfer: The paper strongly advocates for and demonstrates the effectiveness of synthetic data for training robust segmentation models, particularly for tasks like vehicle damage assessment where real data is scarce and expensive to annotate.
  • Procedural Generation for Variability: The procedural approach allows for generating an arbitrarily large dataset with diverse damage types, scales, and appearances, which is crucial for training models that generalize well to unseen real-world scenarios.
  • Human-in-the-Loop for Annotation Efficiency: Using a semi-automatic annotation process with a trained model significantly speeds up the manual annotation of 3D models compared to a fully manual approach.
  • CrashCar101's Superiority: The CrashCar101 dataset, even when smaller than some synthetic alternatives like CGPART, yields better performance due to its more realistic rendering and greater diversity of 3D car models.
  • Few-Shot Learning Advantage: Pre-training on CrashCar101 provides a significant performance boost in few-shot damage segmentation scenarios, indicating a smaller domain gap between the synthetic data and real-world damage data compared to general-purpose datasets like COCO or ImageNet.

Key Results

  • Part Segmentation: Models trained on a combination of real data (Pascal-Part) and synthetic data (CrashCar101-Part) outperform models trained solely on real data across multiple architectures (DeepLabv3, SegFormer). Specifically, CrashCar101-Part alone often outperforms CGPART, a larger synthetic dataset.
  • Damage Segmentation (Few-Shot): Pre-training on CrashCar101 for few-shot damage segmentation on the CarDD dataset yields significantly better results than pre-training on COCO or ImageNet. This translates to improvements of +6.3-17.9% mIoU at 1-shot and +4.4-7.0% at 5-shot.
  • Sim2Real Transfer: The results demonstrate a strong sim2real transfer capability of CrashCar101, showing that models trained on this synthetic dataset can generalize well to real-world damage assessment tasks.

Limitations and Future Work

  • Limitations:
    • The current dataset focuses solely on cars.
  • While realistic, the synthetic nature might still have subtle differences from real-world data that could be exploited by more advanced domain adaptation techniques.
  • The paper primarily focuses on 2D image segmentation, though the 3D modalities are available for future research.
  • Future Work:
    • Extending the procedural generation pipeline to handle a wider variety of incidents beyond typical accident damage (e.g., natural disasters).
  • Developing models that can operate on various object types beyond vehicles.
  • Further exploring the use of the 3D modalities for 3D damage assessment tasks.
  • Investigating more advanced domain adaptation techniques to further close the sim2real gap.

Summary updated

BibTeX
@inproceedings{parslov2024crashcar,
  author = {Jens Parslov and Erik Riise and Dim P. Papadopoulos},
  title = {{CrashCar101: Procedural Generation for Damage Assessment}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2024},
}
Figure