Abstract: We compare the end-to-end training approach to a modular approach in which a system is decomposed into semantically meaningful components. We focus on the sample complexity aspect, in the regime where an extremely high accuracy is necessary, as is the case in autonomous driving applications. We demonstrate cases in which the number of training examples required by the end-to-end approach is exponentially larger than the number of examples required by the semantic abstraction approach.
Introduction
The artificial intelligence community has seen massive success with End-to-End (E2E) training, where a single neural network maps raw sensory input directly to output commands. While E2E is simpler to implement because it avoids manual decomposition or sub-module labeling, it faces a significant "validation bottleneck" in safety-critical applications like autonomous driving.
In these fields, failure rates must be incredibly low—often cited as one failure in millions of hours of operation. To statistically prove that an E2E system has reached such a low error rate ($\epsilon$), standard learning theory (PAC learning) suggests we need a number of samples proportional to $1/\epsilon$. For high-stakes safety, this number becomes practically impossible to collect. This paper provides a theoretical justification for why modular, "semantically abstracted" systems are superior for reaching these extreme reliability targets.
The Validation Bottleneck
For an E2E system, it is mathematically impossible to distinguish between a "good" system and a "slightly less good" system without observing at least $1/\epsilon$ failures. If the target failure rate is $10^{-9}$, billions of data points are required just to verify the system's safety. This makes E2E training "data-hungry" not just for the learning process itself, but for the fundamental task of proving the system is safe for public roads.
Semantic Abstraction and the Modular Approach
To overcome this bottleneck, the authors propose Semantic Abstraction. In this approach, the system is broken into modules with clear semantic meanings, such as vehicle detection, lane sensing, and path planning. This decomposition allows engineers to leverage "approximate independence."
By measuring the failure rates of individual sub-components rather than the whole "black box," we can bound the total system failure probability. If a system fails only when multiple sub-modules fail simultaneously (acting as an "AND" gate for failures), the total failure probability decreases exponentially with the number of modules. For example, if a car is missed by both a camera-based system and a radar-based system, the probability of a total system failure is the product of their individual failure rates.
Key Theoretical Insights
- The Power of "AND": Mathematically, multiplying small probabilities leads to a tiny total probability that can be verified by testing individual components much more easily than testing the integrated whole.
- Prior Knowledge as a Data Substitute: Modular design allows engineers to inject prior knowledge (e.g., "it is easier to see a car nearby than one far away"). This reduces the amount of raw data needed to reach a high confidence level.
- Exponential Advantage: The paper demonstrates that the number of training examples required by the E2E approach can be exponentially larger than those required by semantic abstraction to reach the same level of certified safety.
Conclusion and Limitations
While modularity offers a clear path to validating high-safety standards, it is not without trade-offs. The mathematical advantage relies on sub-modules being "c-approximately independent." If failures are highly correlated (e.g., both camera and radar are blinded by the same specific lighting condition), the exponential benefit disappears. Furthermore, while modularity saves on sample complexity, it increases labeling complexity, as humans must provide ground truth for every sub-module rather than just the final control signal.
Ultimately, the authors argue that for the extreme reliability required by autonomous vehicles, the semantic abstraction approach provides a necessary theoretical and practical advantage over pure end-to-end learning.
Summary updated