Dashcam accident-anticipation models are trained to raise an alarm before a crash. For years they reported that they could do so three to four seconds before impact. A recent re-evaluation shows that much of this lead time came from alarms that fired before any hazard was visible in the frame. Once evaluation counts only alarms raised after the hazard appears, and caps false alarms at 10%, the honest warning horizon shrinks to under one second. We propose a model that can only raise an alarm through a small set of named hazard concepts, such as "pedestrian stepping off the curb" or "lead vehicle braking hard," that the system detects frame by frame. The model is not allowed to react directly to the overall look of a scene. If a concept is not active, the alarm cannot fire. Every alarm can therefore be traced to, and corrected through, a readable cause.
The bet is that this constraint does more than make the model readable. It should also cut the specific failure that inflated the old numbers. A model that fires before the hazard appears is reacting to something other than the hazard, for example the visual style of crash-compilation footage. A concept bottleneck (an intermediate layer forced to predict human-named concepts, through which all information to the output must pass) removes that route by construction. If the bet holds, interpretability helps accuracy under the evaluation that matters for safety. That is a sharper and more checkable version of the usual claim that interpretable models cost little.
Core Claim
Under the strict protocol of Accident Anticipation via Temporal Occurrence Prediction, we expect three results. A temporal concept-bottleneck model with a transparent alarm head should produce fewer premature alarms, meaning alarms raised before the annotated "anomaly appears" timestamp, than an unconstrained encoder with the same backbone and the same supervision. It should do this while matching that encoder's mean time-to-accident at false-alarm rate $\le 10\%$. Correcting a handful of concept values at test time should then measurably move the operating point.
The claim is specific to this protocol and to premature alarms. It is not a general claim that bottlenecks raise AUC.
Relation to Prior Work
The work closest to ours is Driving through the Concept Gridlock. It builds a concept bottleneck for driving: an LLM-generated concept library, CLIP scores of each frame against concept text, and a Longformer over the score sequence. It matches black-box ResNet and ViT baselines on steering and distance prediction (MAE 0.27 and 0.22 on Comma2k19). That paper supplies our method for building concepts. Its target, however, is nominal control. It never models collisions, never tests concept intervention, and never meets the false-alarm problem.
Three accident models motivate the gap. DRIVE explains its alarms with gaze-like saliency maps. CAP uses text annotations only as a training-time teacher. CAMERA attaches a vision-language model that verbalizes risk after an opaque network has already computed it. None routes the decision through an inspectable bottleneck. All three report 3.6–4.5 s lead times under the loose metric that the TOP paper discredits.
Two papers constrain the design.
- Stop explaining black box machine learning models argues that saliency and post-hoc explanations can be unfaithful. This rules out bolting explanations onto TOP after the fact.
- Concept Bottleneck Models warns that side channels bypassing the concepts silently defeat the bottleneck, and that incomplete concept sets cap accuracy. The same paper also supplies the intervention mechanism and the evidence that bottlenecks resist spurious background cues: 48–49% vs. 62.7% error on CUB+Places. That spurious-cue robustness is the closest analogue to our premature-alarm hypothesis.
Research Program
Concept library. We build a library of roughly 100 hazard concepts; Concept Gridlock found performance peaked near that size. The concepts come from two sources: LLM generation, as in Concept Gridlock, and mining of the fact-effect-reason annotations in CAP-DATA. For example, the reason "the cyclist suddenly crossed" becomes the concept cyclist entering ego lane. Mining the annotations gives weak concept labels per video segment without new human labeling, which addresses the annotation burden noted for bottleneck models.
Concept predictor. A concept predictor $g$ maps each short video snippet to concept probabilities $c_t \in [0,1]^k$, where $t$ indexes snippets and $k$ is the number of concepts. It is trained against the mined labels. For a fair comparison, $g$ uses the same snippet-level 3D convolutional encoder as TOP.
Alarm head. The alarm head keeps TOP's multi-horizon output, a probability that the crash occurs within each horizon $\tau$. It reads only a short window of concept vectors:
$$ p_t(\tau) = \sigma\Big(b_\tau + \sum_{j=0}^{L-1} w_{\tau,j}^{\top} c_{t-j}\Big), $$
where $\sigma$ is the logistic function, $L$ is a window of a few snippets, $b_\tau$ is a bias, and $w_{\tau,j}$ are sparse weights. Every alarm then decomposes exactly into per-concept, per-lag contributions.
Training. We train sequentially (concepts first, then head) and jointly, following the CBM training variants. There is no residual path from video features to the alarm.
Analysis. We measure premature alarms, faithfulness of the concept attributions, and the effect of intervention: overwriting $c_t$ with annotation-derived values and re-scoring.
Baselines
Naive reference. A constant-hazard predictor fires on a fixed schedule tuned to FAR $\le 10\%$. It gives the time-to-accident a model earns with no scene understanding. Any learned model must clear it before its lead time means anything.
Strongest existing method: TOP. This is Accident Anticipation via Temporal Occurrence Prediction, which pairs a snippet 3D-CNN encoder with a multi-horizon Transformer decoder. It is the strongest competitor because it is the only method evaluated under the strict protocol we adopt. It reports AUC 0.6747 at 0.5 s before the crash on CAP, AUC 0.8381 at the collision moment, and mean TTA of 0.8644 s on CAP and 0.8848 s on DADA at FAR $\le 10\%$. We will retrain it on MM-AU's CAP and DADA subsets with the same encoder initialization, the same refined timestamps, the same epochs, and an equal hyperparameter search budget.
We beat TOP if we cut the premature-alarm rate by a clear margin across three seeds while keeping mean TTA at FAR $\le 10\%$ within roughly 0.1 s of TOP's. Matching TTA alone does not count as beating it.
Loose-protocol method: CAMERA. CAMERA reports 89.64% AUC and an mTTA of 4.48 s on DADA-2000, but under the loose protocol. Where we can re-run CAMERA's video-only path, we will report it under the strict protocol. Otherwise we say so and do not compare numbers across protocols.
Ablations that isolate the bottleneck. We run three. The first is a leaky variant that adds a residual feature path to the head. The second keeps the bottleneck but replaces the linear head with a Transformer over concepts, which separates the effect of the bottleneck from the effect of the head's readability. The third is a same-width bottleneck of unsupervised, unnamed units, which tests whether naming matters or mere narrowness does.
Why not just…?
Why not just keep TOP and add saliency maps or post-hoc concept probes? Probes read concepts from a model that never had to use them. If TOP fires prematurely because of a scene-style cue, a probe will still find plausible concepts. The critique in Stop explaining black box machine learning models is precisely that such readings need not reflect the computation. Our faithfulness test settles this directly. We zero a concept's input and check whether the alarm moves as its attribution predicts. We run this test for both our model and a probed TOP.
Why not just gate TOP's alarm on an object detector seeing a road user nearby? This is a real threat. A cheap gate might remove most premature alarms on its own. We include a detector-gated TOP as a baseline. If it matches our premature-alarm reduction and TTA, then the safety benefit comes from requiring visible evidence, not from concepts. Our remaining contributions would then be attribution and intervention, and we would narrow the claim accordingly. We expect the gate to suppress true early warnings driven by non-object precursors such as occlusion or road-surface cues, and the per-crash-type breakdown on DADA's 54 types would show this.
Why not just ask a vision-language model to explain or decide? In CAMERA, the language model verbalizes a decision already made by an opaque network, so its explanation is not the cause of the alarm. Concept Gridlock gets about 0.1 s latency from structured concept scores, which matters at sub-second warning horizons. We will report a zero-shot vision-language baseline's latency alongside its strict-protocol TTA. We found no strict-protocol result on Lacuna for any vision-language accident model.
Why not just train sparse rule lists on kinematics? Dashcam benchmarks provide no reliable kinematics. As the same position paper notes, rule lists need meaningful features extracted first. Our concept layer is that extraction step, and the linear head is already close to a scoring system.
Proposed Experiments and Decisive Checks
All main experiments use the MM-AU CAP and DADA subsets with TOP's refined timestamps. We report AUC at 0.5 s and 0.0 s, mean TTA at FAR $\le 10\%$, and the premature-alarm rate. DADA-2000 crash types allow per-category analysis. DeepAccident's 12 NHTSA-derived scenarios serve as a secondary, simulated check, where precise collision timestamps make it unambiguous whether a hazard was visible before an alarm.
First decisive check: the premature-alarm comparison. If the bottlenecked model shows no fewer premature alarms than TOP at matched FAR, the central claim fails.
Second decisive check: leakage. The leaky variant should recover any TTA we lose while giving back premature alarms. That would confirm the bottleneck is doing the filtering.
Third decisive check: intervention. We replace predicted concepts with annotation-derived ones on the CAP test split. The test passes if correcting only the top two or three mispredicted concepts per clip measurably raises strict-protocol TTA or lowers FAR. This mirrors the OAI result in which correcting two concepts cut error from above 0.4 to about 0.3.
Fourth check: concept quality. We have humans rate whether the top-weighted concepts behind each alarm describe the true precursor, using the protocol behind Concept Gridlock's 94% descriptive accuracy. We also report how concept accuracy degrades in occluded and low-light clips, the known weakness of single-camera models.
References
- Accident Anticipation via Temporal Occurrence Prediction
- Driving through the Concept Gridlock: Unraveling Explainability Bottlenecks in Automated Driving
- DRIVE: Deep Reinforced Accident Anticipation with Visual Explanation
- Cognitive Accident Prediction in Driving Scenes: A Multimodality Benchmark
- Eyes on the Road, Mind Beyond Vision: Context-Aware Multi-modal Enhanced Risk Anticipation
- Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead
- Concept Bottleneck Models