DBHiCo: Dual-Branch Hierarchical Contrastive Learning for MRI Image Classification

Chengsheng Zhanga, Wei Li, Chenghao Sun, Jian Gong, Xinmei TianarXiv 2026 · arXiv

TL;DR

DBHiCo introduces a dual-branch framework for MRI classification that leverages original and skull-stripped scans as a "natural" multi-view pair. By replacing standard binary contrastive loss with a hierarchical, margin-ranked similarity objective, the model captures fine-grained anatomical nuances that traditional deep learning methods often overlook.

What Problem Does This Paper Address?

Automated MRI classification is a cornerstone of modern neurological diagnosis, yet it faces two major hurdles: acquisition noise and high inter-subject structural similarity. Because human brains look remarkably alike, the subtle pathological variations indicative of diseases like Alzheimer’s or pediatric tumors are often buried under noise or general anatomical commonalities.

While Contrastive Learning (CL)—a technique where models learn by comparing similar and dissimilar pairs—has shown promise, the authors argue that current "intra-view" paradigms are too rigid. They typically use a binary partition (same class vs. different class), treating all negative samples as equally different. This ignores the "heterogeneity" of samples; for instance, two different brains with the same type of tumor should be treated differently than a healthy brain and a diseased one. Furthermore, existing methods often fail to enforce structural consistency across different pre-processing views of the same patient.

Methods

The authors propose Dual-Branch Hierarchical Contrastive Learning (DBHiCo), which consists of two primary innovations:

  1. Dual-Branch Architecture: Instead of using random data augmentations (like cropping or flipping) to create "views," DBHiCo uses a natural anatomical pair: the original MRI scan (which preserves holistic structural boundaries) and the skull-stripped scan (which isolates internal brain tissue). Each view is processed by an independent encoder branch to integrate global robustness with localized detail.
  2. Hierarchical Contrastive Loss ($\mathcal{L}_{HiCo}$): This is the core mathematical contribution. Rather than a simple "positive vs. negative" binary constraint, the authors establish a similarity hierarchy across five levels:
  3. Self cross-view: The highest similarity (same patient, different views).
  4. Intra-class/intra-view: High similarity (different patients, same disease, same view).
  5. Intra-class/cross-view: Moderate similarity.
  6. Inter-class/intra-view: Low similarity (different diseases).
  7. Inter-class/cross-view: Lowest similarity.

By using a margin-ranked loss, the network is forced to "rank" these pairs correctly in the latent space, amplifying the discriminative power of the learned features.

Datasets

The model was evaluated on three distinct neurological benchmarks:

  • Tiantan Dataset: A proprietary clinical dataset from Beijing Tiantan Hospital containing 829 cases across four pediatric tumor types (Medulloblastoma, Choroid plexus tumors, Ependymoma, and Glioma).
  • OASIS-1: A widely used public dataset for Alzheimer’s Disease classification.
  • ADHD-200: A challenging multi-site competition dataset for Attention-Deficit Hyperactivity Disorder, involving data from six different imaging centers.

Key Insights and Ideas

  • Skull-Stripping as a Feature, Not Just Pre-processing: Traditionally, skull-stripping is a chore done before training. DBHiCo realizes that the difference between the original and stripped scan contains valuable structural information that can be used to anchor contrastive learning.
  • The "Aha" of Negative Heterogeneity: The paper demonstrates that treating all negative samples as a monolithic group limits a model's ability to learn fine-grained details. By explicitly modeling "how different" two samples are based on their class and view, the model develops a more nuanced "understanding" of brain anatomy.
  • Synergy of Branches: Ablation studies showed that while the dual-branch setup helps, it only reaches its full potential when coupled with the hierarchical loss. The loss provides the "instruction manual" for how the two branches should relate to one another.

Key Results

DBHiCo achieved state-of-the-art (SOTA) results across all tested datasets:

  • Tiantan: Achieved 86.51% Accuracy, significantly outperforming standard baselines like SimCLR (83.43%) and SuperCon (83.51%).
  • OASIS-1: Reached 85.55% Accuracy and 91.13% AUC, surpassing specialized medical contrastive methods like CoCa and BPaCo.
  • ADHD-200: On this notoriously difficult multi-site dataset, DBHiCo achieved an average accuracy of 69.81%, proving its robustness against domain shifts caused by different scanners.
  • Interpretability: Grad-CAM visualizations showed that DBHiCo focuses much more precisely on pathological tissues (like tumor boundaries) compared to baseline models, which often suffered from background interference.

Limitations and Future Work

  • Cross-Sectional Focus: The current study focuses on cross-sectional structural MRI. The authors do not explore longitudinal data (tracking a patient over time) or functional MRI (fMRI), which could provide additional temporal insights.
  • Computational Overhead: While the dual-branch architecture is effective, it effectively doubles the encoder parameters during training (though the authors note it remains efficient enough for standard hardware).
  • Future Directions: The authors suggest that this hierarchical approach could be extended to other medical imaging modalities (like CT or PET) where "natural" multi-view pairs exist through different reconstruction kernels or windowing settings.

Summary updated

Abstract

Automated Magnetic Resonance Imaging (MRI) image classification provides a crucial non-invasive approach for neurological diagnosis, yet it is fundamentally challenged by inherent acquisition noise and high inter-subject structural similarity, which obscure subtle pathological variations, thereby limiting the performance of deep neural networks. While contrastive learning (CL) offers a promising mitigation strategy, existing paradigms rely on rigid binary partitions that ignore negative sample heterogeneity, and they fail to capture the cross-view structural consistency vital for fine-grained anatomical alignment. To address these barriers, we propose Dual-Branch Hierarchical Contrastive Learning (DBHiCo), a novel framework explicitly designed to collaboratively enforce structural consistency and semantic discrimination. Specifically, DBHiCo leverages a natural anatomical multi-view pair (original and skull-stripped MRI scans), processing them through a dual-branch architecture to seamlessly integrate holistic structural robustness with localized intracranial details. Crucially, we introduce a hierarchical contrastive loss that discards conventional binary constraints in favor of a margin-ranked similarity hierarchy. This mechanism explicitly orders positive and negative pairs based on their structural and semantic difficulty, thereby amplifying the fine-grained semantic discriminability of the learned representations. Extensive experiments across three diverse neurological datasets demonstrate that DBHiCo achieves state-of-the-art performance, consistently outperforming existing MRI multi-view representation learning and advanced contrastive fine-tuning methods.
BibTeX
@misc{zhanga2026dbhico,
  author = {Chengsheng Zhanga and Wei Li and Chenghao Sun and Jian Gong and Xinmei Tian},
  title = {{DBHiCo: Dual-Branch Hierarchical Contrastive Learning for MRI Image Classification}},
  year = {2026},
  eprint = {2611.19511},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.19511},
}
Figure