Harnessing spatial homogeneity of neuroimaging data: patch individual filter layers for CNNs

Kerstin Ritter geb. Hackmack, Fabian Eitel, Jan Philipp Albrecht, Martin Weygandt, Friedemann Paul, Kerstin RitterarXiv 2020 · arXiv

Abstract

Standard Convolutional Neural Networks (CNNs) utilize weight sharing to achieve translation invariance, a property essential for general computer vision but potentially suboptimal for neuroimaging. Brain scans are typically spatially normalized to a standard anatomical template, ensuring that specific structures appear in consistent locations across subjects. We propose Patch Individual Filter (PIF) layers, which relax weight-sharing constraints in deeper network layers to exploit this spatial consistency. Our results across three neuroimaging tasks—sex classification, Alzheimer’s disease detection, and multiple sclerosis identification—demonstrate that PIF-based architectures achieve higher accuracy and significantly faster convergence than standard CNNs, particularly in small-sample clinical scenarios.

Introduction: The Case for Spatial Priors

The success of CNNs in computer vision is largely attributed to the assumption of translation invariance: a feature, such as an airplane, should be recognizable regardless of its position in an image. However, neuroimaging data is fundamentally different. Researchers routinely use spatial normalization to align MRI scans to a common coordinate system (e.g., the MNI152 atlas). In this context, anatomical landmarks like the hippocampus or cerebellum occupy fixed spatial regions across the dataset.

Conceptual motivation for PIF layers: General objects (a) exhibit high spatial variance, while normalized neuroimaging data (b, c) is naturally homogenous and spatially consistent.

Applying a global sliding filter across the entire brain is computationally redundant and introduces noise by searching for localized pathologies in unrelated anatomical regions. The authors address this mismatch by introducing a "spatial prior," essentially informing the network that location matters.

Methods: Patch Individual Filter (PIF) Layers

To leverage the structured nature of medical data, the researchers developed PIF layers. Unlike standard convolutions that share weights across the entire spatial domain, PIF layers operate in three phases:

  1. Split: The input feature map is partitioned into a grid of spatial patches (e.g., $5 \\times 5 \\times 5$ voxels).
  2. Process: Each patch is assigned its own unique set of filters. These filters are learned independently, allowing the network to specialize in detecting features specific to that anatomical region without interference from other areas.
  3. Reassemble: The resulting patches are stitched back together to restore the original feature map dimensions.

The proposed architecture is a hybrid: early layers employ standard convolutions to capture low-level, global features (like edges), while the final convolutional layer is replaced with a PIF layer to extract localized, high-level abstract features. To mitigate potential artifacts at patch boundaries, the authors implement a parallel stream using overlapping patches shifted by half the patch size.

Experimental Evaluation

The PIF architecture was evaluated against optimized baseline CNNs across three distinct tasks using high-resolution structural imaging:

  • UK Biobank: 1,854 T1-weighted MRIs for sex classification (Male vs. Female).
  • ADNI (Alzheimer’s Disease Neuroimaging Initiative): 969 T1-weighted MRIs for detecting Alzheimer’s Disease (AD) vs. Healthy Controls (HC).
  • VIMS (Private Hospital Data): 147 FLAIR images for detecting Multiple Sclerosis (MS) vs. Healthy Controls.
Example of a T1-weighted MRI scan (coronal view) similar to those used in the UK Biobank and ADNI datasets.

To simulate the constraints of clinical research, the models were also tested on 20% subsets of the UK Biobank and ADNI datasets.

Key Results

Performance and Small Data Robustness

PIF layers consistently outperformed baseline models, with the most significant gains observed in small-sample regimes. In the UK Biobank sex classification task using only 371 samples, the PIF model achieved a balanced accuracy of 88.41%, an 8.22% improvement over the baseline (80.19%). Similarly, in the MS detection task—the smallest dataset—PIF reached 79.46% accuracy compared to the baseline's 76.35%.

Training Efficiency

PIF models exhibited significantly faster convergence. For the full ADNI dataset, PIF-based networks reached their early stopping criteria in an average of 21.3 iterations, whereas standard CNNs required 34.2 iterations. This suggests that enforcing spatial locality simplifies the optimization landscape, allowing the model to reach peak performance sooner.

Discussion and Limitations

By incorporating a spatial prior directly into the network architecture, PIF layers reduce the superimposition of signal and noise. Visualization via Layer-wise Relevance Propagation (LRP) indicates that while standard CNNs eventually attempt to learn localized features, PIF layers enforce this structure from the outset, leading to a more efficient training process.

However, several limitations remain. The reliance on spatial normalization means PIF layers are not suitable for unaligned or "raw" clinical data. Furthermore, the choice of patch size and overlap introduces new hyperparameters that require task-specific tuning. Future research may explore biologically-informed partitioning based on brain atlases rather than rigid grids to further enhance the model's interpretability.

Conclusion

Patch Individual Filter layers offer a powerful alternative to standard CNNs for spatially structured medical imaging. By relaxing the weight-sharing constraint, these layers harness the inherent spatial homogeneity of neuroimaging data, providing a robust solution for high-performance deep learning in data-constrained clinical environments.

Summary updated

BibTeX
@misc{hackmack2020harnessing,
  author = {Kerstin Ritter geb. Hackmack and Fabian Eitel and Jan Philipp Albrecht and Martin Weygandt and Friedemann Paul and Kerstin Ritter},
  title = {{Harnessing spatial homogeneity of neuroimaging data: patch individual filter layers for CNNs}},
  year = {2020},
  eprint = {2007.11899},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2007.11899},
}
Figure