Abstract
Ensuring validation for highly automated driving poses significant obstacles to the widespread adoption of highly automated vehicles. Scenario-based testing offers a potential solution by reducing the homologation effort required for these systems. However, a crucial prerequisite, yet unresolved, is the definition and reduction of the test space to a finite number of scenarios. To tackle this challenge, we propose an extension to a contrastive learning approach utilizing graphs to construct a meaningful embedding space. Our approach demonstrates the continuous mapping of scenes using scene-specific features and the formation of thematically similar clusters based on the resulting embeddings. Based on the found clusters, similar scenes could be identified in the subsequent test process, which can lead to a reduction in redundant test runs.
Introduction
Validating highly automated driving (HAD) systems is an enormous challenge. The "test space"—comprising every possible combination of weather, traffic, and road geometry—is effectively infinite. Testing every scenario is economically and physically impossible. To solve this, engineers use scenario-based testing, but this requires a way to categorize and count scenarios so they do not waste resources testing the same "type" of situation repeatedly. Traditionally, this categorization is done via manual annotation, which is slow, subjective, and does not scale. This paper addresses the need for an automated, scalable way to measure traffic scene similarity without relying on human labels.
Methodology
The proposed pipeline transforms raw traffic data into a searchable mathematical space through four primary stages:
1. Semantic Scene Graphs (SSG)
Instead of using raw coordinates, the model represents a scene as a graph. Nodes represent traffic participants (cars, bikes, pedestrians) with attributes like velocity. Edges represent their relationships (e.g., "car A is behind car B" or "car C is intersecting car D's path"). This abstraction allows the model to focus on interactions rather than absolute positions.
2. Graph Augmentation
To train the model without labels, a self-supervised approach is employed. The researchers take a scene and slightly modify the speed or position of vehicles to create a "positive" pair—scenes that are semantically identical despite minor variations.
3. Contrastive Learning (Triplet Loss)
A Graph Neural Network (GNN) encodes these graphs into a 12-dimensional embedding space. The training uses a triplet loss function, which forces the model to minimize the distance between an "anchor" scene and its "positive" (augmented) version, while maximizing the distance to a "negative" (randomly chosen) scene.
4. Dimensionality Reduction and Clustering
Finally, the model uses UMAP to project the 12D space into 2D for visualization and applies hierarchical clustering to group similar scenes.
Dataset
The study utilizes the INTERACTION motion dataset, which contains diverse traffic scenarios recorded via drones. The scope includes 9,306 scenes across diverse road types, such as roundabouts, merging lanes, and intersections, from both Germany and the USA. The data includes high-definition (HD) maps and object tracks sampled at 10 Hz.
Key Insights
- Relational over Absolute: By using graphs, the model ignores whether a scene happens in Munich or Los Angeles. It prioritizes the constellation of vehicles, allowing it to find "thematically similar" scenes across entirely different road geometries.
- The "Ordinal" Embedding Space: The embedding space organized itself logically. When visualized, the model created distinct "lines" of data points that corresponded to the number of vehicles in a scene (1 car, 2 cars, 3 cars, etc.), even though it was never explicitly told to count participants.
- Semantic Continuity: The embedding space shows gradients rather than random scattering. For instance, scenes with high average speeds naturally cluster at one end of the space, while low-speed scenes cluster at the other.
Results
The model achieved a 99.3% accuracy in correctly identifying that a positive sample was closer to the anchor than a negative sample. To prove the embeddings contained useful information, the authors trained a small Multi-Layer Perceptron (MLP) to estimate scene features based only on the 12D embedding. The model could estimate the number of cars with a deviation of less than one car on average. Qualitatively, the researchers successfully identified 25 distinct clusters. For example, one cluster consistently grouped scenes with long queues of traffic participants, regardless of the specific road or country of origin.
Conclusion and Future Work
While the model avoids manual labels, defining "ground truth" similarity for complex scenes involving dozens of vehicles remains a technical challenge. However, because this encoder creates a meaningful latent space, it could be used in the future for generative models. Instead of just finding similar scenes, researchers could "sample" the space to generate new, synthetic edge cases for testing. Furthermore, this method could identify "gaps" in existing training datasets; if a certain area of the embedding space is empty, it indicates the autonomous system has not been sufficiently exposed to that specific type of traffic interaction.
Summary updated