Attribution Through the Decoder: Prediction-Space Explanations for Structured Outputs

Zaer Bin Zaqy, Brinnae BentarXiv 2026 · arXiv

TL;DR

Standard attribution methods often fail for structured outputs (like angles or 3D rotations) because they explain raw network coordinates rather than the final decoded prediction. By attributing through the decoder differential, this paper introduces "prediction-space" explanations that ignore irrelevant changes (like vector scaling) and significantly improve the faithfulness of feature heatmaps.

What Problem Does This Paper Address?

When a neural network predicts a gaze direction, an angle, or a 3D rotation, it doesn't output these values directly. Instead, it emits a vector of raw coordinates ($z$) which a fixed, parameter-free decoder ($\pi$) maps to the final structured prediction ($y = \pi(z)$). For example, a gaze model might output a 3D vector and then normalize it to get a direction on a sphere.

The authors identify two major failure modes in current attribution methods (like Grad-CAM or Integrated Gradients) when applied to these models:

  1. Radial Contamination: The decoder often discards certain information. In a normalization decoder, the length of the output vector is ignored—only the direction matters. Standard attribution methods still credit inputs that change this length, even though those inputs have zero effect on the final prediction.
  2. Scalarization Loss: Most attribution methods require a single scalar target. If you just sum the output coordinates, you only capture sensitivity in one specific direction, potentially missing inputs that move the prediction in other, equally important directions.
Figure 1. Left: a structured-output model factors as x 7→z 7→π(z); attribution is normally taken with respect to z, but the quantity to be explained is the decoded prediction y. Right: on S1, z and 2z denote the same prediction, so variation along the ray is discarded by the decoder; tangential variation moves the prediction, radial variation spans ker Dπ and cannot. By the normalisation Jacobian (§2.1) the tangent projector is the decoder differential up to a positive scalar.

Methods

The core proposal is to attribute through the decoder differential. Formally, if the model is $x \mapsto z = f(x) \mapsto y = \pi(z)$, the explanation should use the chain rule: $D(\pi \circ f) = D\pi \cdot Df$.

For common normalization decoders, this operation reduces to a tangent projection. The authors adapt several popular attribution techniques into "tangent" or "prediction-space" variants:

  • Tangent Grad-CAM / HiResCAM / LayerCAM: These project the gradients onto the tangent space of the output manifold (e.g., a sphere) before generating the heatmap.
  • Angular Integrated Gradients (AIG): A variant of Integrated Gradients that integrates changes in the actual predicted angle rather than raw coordinates.
  • Circular/Spherical LIME: Local surrogate models that use circular or spherical regression targets.

Datasets

The authors evaluate their methods across four distinct settings:

  • Synthetic Oracles: Custom-built tasks where the ground-truth relevance is known. One region of the image controls the output direction (relevant), while another controls only the discarded magnitude (irrelevant).
  • Plant (S1): A circular regression task predicting the time of day from images of plants.
  • Gaze (S2): Gaze direction estimation using the MPIIGaze dataset.
  • Pose (SO(3)): 3D object pose estimation using the PASCAL3D+ dataset, employing both quaternion and 6D rotation decoders.

Key Insights and Ideas

  • The "Invariance" Requirement: An explanation should be "decoder-invariant." If two different raw output vectors $z$ and $2z$ map to the same prediction $y$, an input that only moves the model from $z$ toward $2z$ should receive zero credit.
  • Ambient Attribution can be "Below Chance": On synthetic tasks, standard "ambient" attribution assigned up to 72% of its mass to pixels that were provably irrelevant to the prediction.
  • Representation Sharing: The failure of standard methods is most acute when the network's internal representations for the "relevant" and "discarded" information are highly separated. As the pathways share more features, the "gap" between ambient and tangent attribution shrinks, though it rarely disappears.
Figure 2. Controlled representational separability. Left: attribution mass on the angle-encoding region against α, the amount of prediction- relevant representation mixed into the decoder-null pathway (5 seeds per point, mean ± SD). Tangent is flat at 1.000 by construction; radial climbs off zero as the null pathway begins reading region-A features; ambient, a mixture, follows. The shaded band is the gap. Right: the same gap against the mean cosine similarity of tangent and radial saliency maps over the first 40 test images per run (pooled Pearson r = −0.76; definition in Appendix I.1). All 35 runs are shown.
  • Scalarization is a separate bottleneck: Simply making a scalar target decoder-invariant (e.g., using the sum of a rotation matrix) isn't enough. On 3D pose tasks, the full differential significantly outperformed invariant scalar targets, proving that capturing the multi-dimensional nature of the prediction is critical.

Key Results

  • Gaze Estimation: Tangent Grad-CAM produced a 44% greater prediction displacement than the ambient baseline when the top 10% of attributed pixels were deleted, indicating it found much more influential features.
  • 3D Pose (6D Decoder): Using the full differential improved mean deletion displacement by 14.1° over the standard coordinate sum.
  • Synthetic Tasks: Tangent methods achieved near-perfect Oracle Attribution Mass Ratios (AMR $\approx 1.0$), while ambient methods hovered near or below chance (0.5).

Limitations and Future Work

  • Mixed Results for LayerCAM: While tangent variants generally helped, the results for a two-polarity tangent LayerCAM were mixed, showing gains in circular regression but no detectable difference in quaternion pose.
  • Invariance $\neq$ Faithfulness: The authors warn that while decoder invariance is a necessary property for a "correct" explanation, it doesn't guarantee the explanation will be useful. An invariant method can still miss important features; it just won't highlight provably irrelevant ones.
  • Complexity: Some variants (like the full differential for 6D pose) require multiple backward passes, increasing computational cost compared to simple gradient-based methods.
  • Future Directions: The authors suggest extending this framework to more complex structured outputs and investigating how these "prediction-space" heatmaps can better help practitioners debug models in the wild.

Summary updated

Abstract

Networks that predict angles, gaze directions, and rotations do not output these quantities directly: they emit raw coordinates that a fixed decoder, such as normalization, maps to the final prediction. Attribution methods are usually applied to the raw coordinates, so they can credit inputs for changes the decoder discards, such as rescaling a vector when only its direction matters. The resulting explanation can highlight inputs that have no effect on the prediction. We instead attribute through the decoder differential, which for normalization decoders reduces to a scaled tangent projection. We characterize invariance to output rescaling, show that every fixed smooth scalar target on a sphere has points of zero tangential sensitivity, and show that an endpoint-projected variant of Integrated Gradients can still attribute to scale-only inputs, whereas direct angular Integrated Gradients cannot. On synthetic tasks with analytically known relevance, ambient attribution assigns up to 72% of its mass to an irrelevant region. On MPIIGaze validation data, tangent Grad-CAM produces 44% greater prediction displacement under 10% pixel deletion. On PASCAL3D+ with a 6D rotation decoder, the full differential improves mean deletion displacement by 14.1 degrees over the ambient coordinate sum and by 13.8 degrees over a decoder-invariant scalar. Results for a two-polarity tangent LayerCAM variant are mixed: it exceeds ambient-sum LayerCAM on circular regression but no difference is detected on quaternion pose. These results separate two failure sources, radial contamination and scalarization loss. The approach uses existing attribution methods with decoder-aware targets, although some variants also require changes to attribution aggregation. An invariant attribution can still miss inputs that drive the prediction, so invariance and deletion performance should each be checked.
BibTeX
@misc{zaqy2026attribution,
  author = {Zaer Bin Zaqy and Brinnae Bent},
  title = {{Attribution Through the Decoder: Prediction-Space Explanations for Structured Outputs}},
  year = {2026},
  eprint = {2611.23161},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.23161},
}
Figure