Interpretable-by-design models often route predictions through a "concept bottleneck": a layer of human-named attributes, such as "wing color" or "smiling," that a person can read and edit. To recover the accuracy these bottlenecks lose, recent models add a sidechannel. This is an unlabeled latent vector that lets extra information bypass the concepts on its way to the predictor. The catch is that gradient descent tends to drift toward solutions that ignore the concepts and lean on the sidechannel. Quantifying the Accuracy-Interpretability Trade-Off in Concept-Based Sidechannel Models calls this the "accuracy trap": "Models trained solely to maximize accuracy naturally gravitate toward low SIS scores." SIS, the Sidechannel Independence Score, measures how much the prediction depends on concepts rather than the bypass. That paper treats the resulting accuracy-versus-SIS curve as the trade-off itself, and it controls the curve with a penalty weight $\beta$.
This proposal asks whether that curve describes the objective or the optimizer. When the concepts carry enough information to predict the label, the training loss typically has many equally good minimizers. Some rely on the concepts and some rely on the sidechannel, and gradient descent's implicit preferences, not the loss, decide which one training reaches. If that is right, part of the published accuracy-interpretability trade-off is a choice made by the optimization path. A different path could then recover concept reliance at no cost in accuracy. When the concepts are incomplete, the trade-off should be real, and no change of path should beat it. Separating these two cases, in theory and on standard benchmarks, is the contribution.
Core Claim
The claim has two parts. First, in sidechannel models whose concepts are complete, the accuracy trap is not a bad local minimum. It is gradient descent selecting one point from a large connected set of equal-loss minimizers, and the selected point is the one that uses the sidechannel. Second, this can be detected empirically. Train from different starting points or along a different path at the same $\beta$. If concepts are complete, these runs should reach solutions with the same accuracy and higher SIS, joined to the trapped solution by a low-loss path. If concepts are incomplete, the higher-SIS solutions should cost accuracy.
The framing comes from the stationary-point literature. Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Łojasiewicz Condition shows that under the Polyak-Łojasiewicz (PL) inequality every stationary point is a global minimum. The PL inequality is a condition requiring the gradient to be large wherever the loss is far from its minimum. That result constrains the story: if PL holds approximately, as it often does in overparameterized models, the trap cannot be a spurious local minimum. It must instead be a question of which global minimum training reaches. We therefore do not search for bad stationary points. We characterize the geometry of the minimizer set and the path dependence of the point gradient descent selects within it.
Research Program
Theory on a model we can solve. The first part is analysis of a two-path model. A concept head maps inputs to $k$ supervised concepts through a sigmoid, a linear sidechannel maps inputs to $m$ latents, and a linear predictor reads both. Training minimizes
$$\mathcal{L}(\theta) = \mathcal{L}_{task}(\theta) + \lambda\,\mathcal{L}_{concept}(\theta) + \beta\,\mathcal{L}_{SIS}(\theta),$$
where $\theta$ collects all weights, $\lambda$ weights concept supervision, and $\beta$ weights the SIS penalty of Debot and Marra.
We aim to show three things. The first is that, for complete concepts, the zero-task-loss set at $\beta=0$ is a manifold that contains both concept-reliant and sidechannel-reliant points. The second is that gradient flow from small initialization reaches the sidechannel end of this manifold. The intuitive mechanism is that the linear path's gradients are not attenuated by sigmoid saturation, nor pulled toward the concept labels by the concept loss. The third is a threshold $\beta^*$ above which the sidechannel end stops being stationary.
To check that conclusions about empirical stationary points carry over to the population loss, we reuse the vector-valued Rademacher and gradient chain-rule tools of Uniform Convergence of Gradients for Non-Convex Learning and Optimization. The method comes from that paper; we apply it to a new composed model. Its own caveat binds us: for ReLU components the guarantees become dimension-dependent, so the theory uses smooth activations, and the deep experiments are the test of whether the picture survives.
A diagnostic for deep models. The second part is a protocol that separates "objective trade-off" from "optimizer trade-off" in real networks. We fix $\beta$ and train concept sidechannel models from three kinds of start. The first is random initialization. The second is a warm start from a converged sequential concept bottleneck, with the sidechannel then switched on. The third is a continuation schedule that begins at large $\beta$ and lowers it gradually to the target value.
For every endpoint we record five quantities. These are task accuracy, SIS, the final gradient norm (to confirm approximate stationarity), the smallest Hessian eigenvalue of the predictor and bottleneck layers estimated by Lanczos iteration (to separate minima from saddles), and the loss barrier along linear and Bézier interpolation paths between endpoints. Evidence for an optimizer artifact would be same-loss, different-SIS endpoints that are all approximate minima and are joined by low barriers.
The fix, if the diagnosis holds. If endpoints of this kind exist, the continuation schedule becomes a training method in its own right. We would also check whether it beats a generous $\beta$ grid search on the accuracy-SIS Pareto front.
Baselines
The naive reference point is the unregularized concept sidechannel model ($\beta=0$), trained end to end with Adam. This is the model Debot and Marra report as gravitating toward low SIS. A black-box network with the same backbone sets the accuracy ceiling.
The strongest existing method for controlling the trade-off is SIS regularization from Quantifying the Accuracy-Interpretability Trade-Off in Concept-Based Sidechannel Models. We will run it with a $\beta$ sweep whose total GPU-hours equal those spent on all our warm-start and continuation runs combined, using the same backbone, data splits, and epochs. No reported SIS or accuracy value for this method was found on Lacuna, so we will reproduce it rather than quote it.
The second strong competitor is the Concept Embedding Model with RandInt training, from Concept Embedding Models: Beyond the Accuracy-Explainability Trade-Off. RandInt stochastically substitutes ground-truth concepts during training to force the model to depend on them. On CelebA its authors report "5% higher task accuracy than a standard black-box model trained without concepts." It attacks the same symptom through a stochastic change to the training path, so it is the closest competitor to our continuation method.
On CUB, the accuracy reference is Coop-CBM with the Concept Orthogonal Loss from Auxiliary Losses for Learning Generalizable Concept-based Models, at "84.1% accuracy," against a black box at "82.3%." For a plain bottleneck, Concept Bottleneck Models reports a joint-CBM "19.9% error rate."
The key ablation is the sequential warm start with no continuation. It isolates whether the benefit comes from changing the path at all or from the specific gradual annealing of $\beta$.
Beating the baselines means a Pareto front of (accuracy, SIS) that weakly dominates the equal-budget $\beta$ sweep, with an SIS gain at matched accuracy that exceeds the seed-to-seed spread over five seeds. It must also show equal or better intervention response, meaning the drop in task error when a fixed number of concepts are set to their true values.
Why not just…?
Why not just tune $\beta$ more carefully? This is the real threat, because if a fine enough grid finds every good solution, there is nothing left to explain. Our claim is specific: at a fixed $\beta$, different paths reach different endpoints. A grid search over $\beta$ alone cannot reveal that, because each grid point still uses the default path. The equal-budget comparison settles the question. If the $\beta$ sweep's front matches the continuation front, we report that the trap is fully controlled by the objective, and the diagnostic still stands as a negative finding.
Why not just train sequentially, concepts first? Concept Bottleneck Models shows that sequential pipelines lose accuracy when concepts are incomplete. With a sidechannel switched on afterward, however, a sequential warm start may be all the fix needs. We include it as an explicit arm and do not argue against it. If it matches continuation, the contribution narrows to the explanation and the completeness test, and we say so.
Why not just use RandInt-style concept substitution? RandInt changes the path stochastically, and its convergence behavior is unanalyzed. It may escape the trap for the same reason our method does. We run it under the same budget and apply the same endpoint diagnostics to its solutions. If RandInt endpoints also sit in the high-SIS part of a connected minimizer set, that unifies the two methods rather than defeating ours.
Proposed Experiments
The decisive manipulation is concept completeness, which we control directly. On MNIST-Addition, used by both Deep concept reasoning: beyond the accuracy-interpretability trade-off and the sidechannel paper, the digit concepts are complete for the sum. We then progressively hide digits from the concept set so that the concepts become incomplete. On CelebA we drop subsets of annotated attributes. On CUB and the CUB + Places shift benchmark, the 112 attributes are naturally incomplete. On the shift benchmark the reference figures are a black box at "27.7%" and Coop-CBM+COL at "36.2%" accuracy, from Auxiliary Losses for Learning Generalizable Concept-based Models. That benchmark tests whether recovered concept reliance buys robustness.
The prediction that could falsify the claim is this. Path-induced SIS gains at fixed $\beta$ and matched accuracy should be large when concepts are complete and should shrink toward zero as concepts are removed. In the incomplete regime, any SIS gain should come with an accuracy loss. Interpolation barriers between trapped and untrapped endpoints should be near zero when concepts are complete and should grow as concepts are removed.
If gains appear even with incomplete concepts, the "equal-loss minimizer set" explanation is wrong, and the effect must come from something else, such as generalization differences. If no gains appear even with complete concepts, the trap is a property of the objective, and the Debot-Marra trade-off curve stands as a fundamental limit.
References
- Quantifying the Accuracy-Interpretability Trade-Off in Concept-Based Sidechannel Models
- Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Łojasiewicz Condition
- Uniform Convergence of Gradients for Non-Convex Learning and Optimization
- Concept Embedding Models: Beyond the Accuracy-Explainability Trade-Off
- Auxiliary Losses for Learning Generalizable Concept-based Models
- Concept Bottleneck Models
- Deep concept reasoning: beyond the accuracy-interpretability trade-off