A lung-nodule classifier that reports "82% malignant" is only useful to a radiologist if it can show which part of the CT volume produced that number. It must also show that removing that part would change the answer. Today's heatmaps fail this test. On the LIDC-IDRI lung CT benchmark, standard attention and attribution methods put only about 3–7% of their highlighted region on the actual clinical feature. The method that fixes this, causal alignment through counterfactual generation, does so with an inner optimization loop whose cost has kept it confined to 2D slices. Lung nodules are 3D objects, and reading them slice by slice discards exactly the out-of-plane shape cues (spiculation, lobulation, vessel attachment) that separate cancer from benign tissue.
We propose to build the diagnostic network as a multi-layer convolutional sparse coder. Such a model explains a volume as a sparse sum of learned 3D patterns, called atoms, stacked in layers so that deep atoms are built from shallower ones. Because the model can regenerate the input from its own codes, the voxels responsible for a diagnosis can be computed exactly by regenerating only the decision-relevant atoms. A counterfactual becomes simple subtraction: delete those atoms, regenerate the volume, and run it back through the network. This gives causal, volumetric, in-lesion explanations without the bilevel optimization of prior work. The open question is whether such a model still classifies well and whether its subtractive explanations are actually faithful.
Core Claim
Papyan et al. showed that a convolutional network's forward pass is a one-shot approximation to inferring sparse codes under a multi-layer convolutional sparse coding (ML-CSC) model, in which an image is $X = D_1\gamma_1$, $\gamma_1 = D_2\gamma_2$, and so on, with $D_l$ a convolutional dictionary and $\gamma_l$ a sparse code. Sulam et al. turned iterative solvers for this model (ML-ISTA and ML-FISTA) into recurrent networks that share dictionary weights across iterations. What neither work exploited is that the deepest code $\gamma_L$ comes with a linear route back to voxel space. If a classification head assigns a decision weight $s$ to each deepest-layer atom (for example, the head weight times the code's gradient), then the voxel-space evidence for the decision is
$$E = D_1 D_2 \cdots D_L \,(s \odot \gamma_L),$$
where $\odot$ is elementwise product. $E$ is a 3D volume of the same size as the input. Computing it costs one transposed pass through the dictionaries, and differentiating through it is cheap.
We claim three things. First, training with a penalty on evidence outside the expert nodule mask $M$,
$$\mathcal{L}_{\text{ev}} = \frac{\lVert (1-M)\odot E\rVert_1}{\lVert E\rVert_1},$$
yields explanations whose subtractive counterfactuals flip the diagnosis when in-mask atoms are removed and leave it unchanged when out-of-mask atoms are removed. Second, this works at a small fraction of the compute of implicit-gradient counterfactual alignment. Third, using genuinely 3D atoms produces explanations that are continuous across slices, whereas the same model with 2D slice-wise atoms produces fragmented, slice-inconsistent evidence. The second and third claims are testable, and the third makes volumetric context loss a measurable property of explanations rather than only of segmentations.
Positioning Against Prior Work
The proposal draws its method from the ML-CSC line. It reuses the unfolded ML-FISTA solver of Sulam et al. and the stability results of Papyan et al., which tie local sparsity to recovery guarantees. Both were evaluated only on MNIST, SVHN, CIFAR-10, and synthetic data, and neither used the synthesis path as an explanation. The closest medical use, MC-CDic, unfolds proximal gradient descent with multi-scale convolutional dictionaries for multi-contrast MRI reconstruction. It is 2D, it is scored on PSNR and SSIM rather than diagnosis, and its codes are never linked to clinical findings. We keep its multi-scale dictionary idea, move to anisotropic 3D atoms, and attach a diagnostic head.
Two papers motivate the work by showing the gap. Learning Causal Alignment for Reliable Disease Diagnosis argues that attention overlap is merely associational. It aligns minimal label-flipping perturbations with expert regions through an image → attributes → diagnosis hierarchy, using the implicit function theorem and conjugate gradient to differentiate an inner counterfactual loop. That work reports CAM precision rising from roughly 7% for prior methods to 75.1% on LIDC, but on 2D crops and with heavy compute. We keep its evaluation logic and its attribute hierarchy, and replace its optimized counterfactual with a synthesized one. Patrício et al. penalize concept activations outside a segmented lesion with a Dice-style coherence loss in 2D dermoscopy. Our $\mathcal{L}_{\text{ev}}$ is close in spirit, but it acts on a voxel-space reconstruction rather than on a feature-map activation, and that difference is exactly what we test.
On the volumetric side, Roth et al. showed that independent slice predictions break anatomical continuity, and that 3D smoothing of slice outputs raised pancreas Dice from 56% to 68%. PN-SAMP shows that a compact 3D network (1.5M parameters) can jointly predict nodule masks, nine radiologist attributes, and malignancy, with malignancy driven by the attributes. We adopt PN-SAMP's task structure but replace its black-box encoder with unfolded pursuit, because PN-SAMP offers no faithfulness check on its attributes.
One paper constrains the evaluation. Zhang et al. show that LIDC-IDRI consensus malignancy scores are biased toward malignant, with over half of score-4 nodules pathologically benign, and propagate biopsy labels from SCH-LND. Any accuracy claim on raw consensus labels is therefore suspect, so we report on both label sets.
Finally, sparse autoencoders on vision-language encoders learn monosemantic features post hoc. They are the natural alternative source of "atoms" and serve as a baseline below.
Research Program
The model works on 64³ nodule-centered CT patches resampled in-plane, with native slice spacing kept. Atoms are anisotropic (for example 5×5×3 voxels at layer 1) to respect 2.5 mm slices rather than interpolating to isotropy. The encoder is a three-layer ML-CSC unfolded for K iterations of ML-FISTA with shared weights. Proximal steps are soft-thresholding, so each code's sparsity remains meaningful; we do not use the learned ResNet proximal blocks of MC-CDic, which would break the exact synthesis path.
The deepest code feeds a linear attribute head for the nine LIDC attributes, and a linear malignancy head reads the attributes. Following Liu et al.'s hierarchy, evidence maps are computed for both attributes and malignancy. Training combines four terms: reconstruction, sparsity, classification, and $\mathcal{L}_{\text{ev}}$.
Faithfulness is checked by a round trip, not assumed. Pursuit is nonlinear, so deleting atoms and re-encoding need not delete exactly those atoms. Our core measurement therefore re-encodes the edited volume and checks the real network's prediction. We also measure how much of the decision is carried by the reconstruction residual $X - D_1\gamma_1$, which the explanation does not cover.
Because explanations live in voxel space at every layer, we can show the radiologist both levels. Layer-1 synthesis shows edge and texture evidence, such as margin sharpness. Layer-3 synthesis shows shape-level evidence, such as lobulation, which by construction spans several slices.
The main risks are known. Errors compound across layers in deep sparse pursuit, as noted for stacked models by He et al.. Memory for 3D iterative unfolding is also a concern. We address both by keeping depth at three layers and patches small, and by reporting accuracy as a function of K.
Baselines
The naive reference is a 2D ResNet applied slice by slice, with Grad-CAM explanations stacked into a volume. This is what most deployed explanation pipelines do, and it sets the floor for both accuracy and slice continuity.
The strongest accuracy competitor is PN-SAMP, a multi-task 3D CNN, reimplemented on our data splits and patch size with Grad-CAM explanations. Its paper reports 97.58% malignancy accuracy, 89.33% attribute accuracy, and 73.89% Dice on LIDC consensus labels. Given Zhang et al.'s findings, we treat that accuracy as partly a measure of consensus bias. Beating PN-SAMP means matching its accuracy within one standard error on consensus labels, and matching or exceeding its AUC on SCH-LND-propagated labels, while substantially improving explanation faithfulness.
The strongest explanation competitor is causal alignment, run from its released code (causal-alignment code). We run it in its native 2D crop setting, where we expect to reproduce the reported 75.1% CAM precision and 72.2% accuracy on LIDC. We also extend it to a 3D CNN on the same 64³ patches, with the same GPU budget and hyperparameter search budget as our model, and we report its wall-clock time and memory. Beating it means equal or better in-mask evidence precision and counterfactual flip asymmetry in 3D at clearly lower compute. Alternatively, it means succeeding where its 3D extension cannot fit the budget. If no reported 3D number exists, we will say so: we found no reported result on Lacuna for causal alignment on 3D CT volumes.
Three ablations isolate the key ingredients. First, the same unfolded model with 2D slice-wise atoms tests whether volumetric atoms matter. Second, the same model with K = 1, which by Papyan et al.'s equivalence is just a CNN, tests whether iterative pursuit matters. Third, the same model with Patrício-style coherence applied to Grad-CAM instead of $\mathcal{L}_{\text{ev}}$ tests whether voxel-space synthesis matters.
Why not just…?
Why not just add a coherence loss to a 3D CNN? This is the cheapest alternative, and it may well raise in-mask heatmap overlap. Liu et al.'s argument, however, is that overlap does not imply causation: a network can highlight the nodule while deciding from elsewhere. Ablation three tests this directly. If Grad-CAM coherence on a 3D CNN matches our round-trip flip asymmetry, the sparse-coding machinery is unnecessary and we will say so.
Why not just run causal alignment in 3D? This is the real threat to our compute claim. On small 64³ patches, implicit-function-theorem gradients with conjugate gradient may simply fit on a modern GPU. We run it under the same budget. If it fits and matches our faithfulness, our contribution narrows to speed and voxel-space multi-layer explanations, and we will report it that way.
Why not just train a sparse autoencoder on a 3D CNN's features? A post-hoc sparse autoencoder yields sparse, possibly cleaner features. It offers no exact path from a feature back to voxels, though, so counterfactuals still require optimization or attribution. We train one on PN-SAMP features and compare its feature-occlusion flip rates with our subtractive counterfactuals.
Why not just smooth slice-wise outputs in 3D, as Roth et al. did? Smoothing makes explanations look continuous without making them come from 3D structure. Our slice-continuity metric is computed alongside the flip test: smoothed 2D maps that look continuous but fail the subtraction test count against the method.
Proposed Experiments
All experiments use LIDC-IDRI nodules with expert masks and the nine attributes, split by patient. Malignancy is evaluated against consensus labels and against biopsy-informed labels propagated from SCH-LND following Zhang et al. We found no reported result on Lacuna for any method on these relabeled splits, so all methods are retrained. A 2D sanity run on CBIS-DDSM checks that our counterfactual matches causal alignment where the latter is known to work.
Diagnostic metrics are malignancy accuracy and AUC, plus attribute accuracy. Explanation metrics are three. Evidence precision is the fraction of $|E|$ or of heatmap mass inside $M$, directly comparable to Liu et al.'s CAM precision. Flip asymmetry is the rate at which removing in-mask decision atoms flips the re-encoded prediction, minus the rate for removing an equal-norm set of out-of-mask atoms. Slice continuity is the Dice between explanation masks on adjacent slices, together with the fraction of evidence in the largest 3D connected component. Cost is measured in GPU-hours and peak memory.
The decisive check is a joint one. The 3D sparse-code model must reach evidence precision comparable to causal alignment's 75.1%, show clearly positive flip asymmetry after a true round trip, and beat the 2D-atom ablation on slice continuity, all without losing more than a standard error of accuracy against PN-SAMP on biopsy-informed labels. If the round trip fails, meaning re-encoding an edited volume does not reproduce the intended code change, then the synthesis path is not a faithful explanation and the central claim is refuted.
References
- Convolutional Neural Networks Analyzed via Convolutional Sparse Coding
- On Multi-Layer Basis Pursuit, Efficient Algorithms and Convolutional Neural Networks
- Deep Unfolding Convolutional Dictionary Model for Multi-Contrast MRI Super-resolution and Reconstruction
- Learning Causal Alignment for Reliable Disease Diagnosis
- Coherent Concept-based Explanations in Medical Image and Its Application to Skin Lesion Diagnosis
- Deep convolutional networks for pancreas segmentation in CT imaging
- Joint learning for pulmonary nodule segmentation, attributes and malignancy prediction
- Re-thinking and Re-labeling LIDC-IDRI for Robust Pulmonary Cancer Prediction
- Sparse Autoencoders Learn Monosemantic Features in Vision-Language Models
- Unsupervised Feature Learning by Deep Sparse Coding