Interpretable Deep Tracking

Benjamin Thérien, Krzysztof CzarneckiarXiv 2022 · DOI · arXiv

Abstract

As autonomous driving systems (ADS) advance toward higher levels of autonomy, the inherent "black box" nature of deep neural networks (DNNs) presents significant challenges for safety validation and legal accountability. This paper introduces a multi-object tracking (MOT) architecture that is interpretable by design. By aligning high-performance DNNs with handcrafted structural causal models (SCMs) through Interchange Intervention Training (IIT), the proposed system provides human-understandable explanations for tracking decisions—such as distinguishing between an occluded vehicle and a false positive—while maintaining the benefits of end-to-end deep learning.

Introduction

In the context of autonomous perception, performance alone is insufficient. When a self-driving vehicle encounters a critical failure or a near-miss, engineers and regulators must understand the underlying cause. Current state-of-the-art trackers often rely on complex neural architectures that lack transparency. Existing interpretability methods are typically post-hoc or involve distilling complex models into simpler, less capable ones. This research seeks a middle ground: an inherently interpretable, high-performance tracker that follows a rigorous internal reasoning process.

Methodology

The proposed framework adopts the tracking-by-detection paradigm but introduces a structured internal logic to guide the network's reasoning.

System Architecture

The system utilizes a LiDAR-based 3D detector backbone and a Graph Neural Network (GNN) to aggregate information across tracks and detections. The final assignment is performed via the Hungarian algorithm, but the internal features are constrained to follow specific causal paths.

Overview of the interpretable tracking architecture, showing the alignment between the deep neural network and the structural causal model (SCM).

Nuanced Decomposition and SCMs

Tracking decisions are decomposed into seven distinct categories across three domains:

  • Detection-and-track: Appearance or Bounding Box (BBOX) matches.
  • Detection-only: Newborn tracks or false positive detections.
  • Track-only: Out-of-range, false positive tracks, or occluded tracks.

For each category, a Structural Causal Model (SCM) defines the logical flow from high-level variables (e.g., position, velocity, occlusion maps) to a final decision. For example, the "Occluded Track" decision is governed by a truth table that evaluates whether a track matches a detection, is within the occluded area, or is out of range.

Truth table defining the logic for the 'Occluded Track' decision within the SCM framework.

Interchange Intervention Training (IIT)

The core innovation is the use of IIT to force alignment between the DNN's internal representations and the intermediate nodes of the SCMs. During training, the system performs "interventions" by swapping neural representations between different tracks. If the model is truly aligned with the causal logic, its output must change in the exact manner predicted by the SCM. This ensures the network is not merely achieving high accuracy through spurious correlations but is "thinking" according to the provided causal structure.

Key Insights and Results

Alignment as Interpretability

The primary contribution is the demonstration that alignment can serve as a proxy for interpretability. If a network is proven to be aligned with an SCM, the SCM itself becomes a faithful explanation of the network's behavior. This allows for fine-grained error reporting, enabling the system to specify, for instance, that a failure occurred because an object was incorrectly predicted as out of range rather than occluded.

Online Uncertainty and Counterfactuals

Because the model possesses an internal logical structure, it can perform real-time self-checks. If the SCM (using probed internal values) suggests a decision that contradicts the final network output, the system can flag an "uncertain state." Furthermore, the authors propose a metric for "explanation quality" based on counterfactual estimation, measuring how consistently the aligned networks mirror the SCM's behavior in hypothetical scenarios.

Conclusion and Future Work

While the proposed method offers a significant step toward transparent perception, it is not without limitations. There is no absolute mathematical guarantee of 100% alignment in all edge cases, and the system's reasoning is limited by the completeness of the human-designed SCMs. Future research will focus on integrating these interpretable outputs into downstream planning modules, allowing for safer and more predictable autonomous driving behavior.

Summary updated

Abstract

Imagine experiencing a crash as the passenger of an autonomous vehicle. Wouldn't you want to know why it happened? Current end-to-end optimizable deep neural networks (DNNs) in 3D detection, multi-object tracking, and motion forecasting provide little to no explanations about how they make their decisions. To help bridge this gap, we design an end-to-end optimizable multi-object tracking architecture and training protocol inspired by the recently proposed method of interchange intervention training (IIT). By enumerating different tracking decisions and associated reasoning procedures, we can train individual networks to reason about the possible decisions via IIT. Each network's decisions can be explained by the high-level structural causal model (SCM) it is trained in alignment with. Moreover, our proposed model learns to rank these outcomes, leveraging the promise of deep learning in end-to-end training, while being inherently interpretable.
BibTeX
@misc{therien2022interpretable,
  author = {Benjamin Thérien and Krzysztof Czarnecki},
  title = {{Interpretable Deep Tracking}},
  year = {2022},
  eprint = {2210.01266},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2210.01266},
}
Figure