GRAFNet: Multiscale Retinal Processing via Guided Cortical Attention Feedback for Enhancing Medical Image Polyp Segmentation

Abdul Joseph Fofanah, Lian Wen, Alpha Alimamy Kamara, Zhongyi Zhang, David Chen, Albert Patrick SankoharXiv 2026 · DOI · arXiv

TL;DR

This paper introduces GRAFNet, a novel deep learning architecture for polyp segmentation in colonoscopy images. By drawing inspiration from the human visual system's hierarchical processing, multiscale analysis, and feedback mechanisms, GRAFNet achieves state-of-the-art performance, significantly improving accuracy and reducing false positives compared to existing methods. This work highlights the potential of biologically inspired designs for advancing medical image analysis.

What Problem Does This Paper Address?

Accurate polyp segmentation in colonoscopy is crucial for early cancer detection, but it's a challenging task. Polyps exhibit high variability in shape and size, can closely resemble normal colonic structures like folds and vessels, and often appear as subtle, flat lesions. Current deep learning models often struggle with these challenges due to:

  • Unidirectional processing: They typically process information in a single "bottom-up" direction, missing the iterative refinement that humans use.
  • Weak multi-scale fusion: Difficulty in effectively combining information from different scales to capture both fine details and global context.
  • Lack of anatomical constraints: Models don't inherently understand the specific anatomical nuances of the colon, leading to misclassifications.
  • Static attention: Existing attention mechanisms are often fixed and don't adapt to context or higher-level understanding.

These limitations result in missed polyps (false negatives) and misidentification of normal structures as polyps (false positives), both of which can have serious clinical consequences.

Methods

GRAFNet's core innovation lies in its biologically inspired architecture, mimicking three key aspects of human visual processing:

  1. Guided Asymmetric Attention Module (GAAM): This module is inspired by orientation-tuned neurons in the primary visual cortex (V1). It uses steerable filters to selectively enhance polyp boundaries and textures while suppressing noise. It also incorporates "guided attention" that uses contextual information to focus on anatomically plausible regions.
  2. Multiscale Retinal Module (MSRM): This module replicates the parallel pathways of the retina (parvocellular, magnocellular, koniocellular, and ON-OFF pathways). This allows for simultaneous analysis of texture, shape, and color contrast, reducing redundancy and enabling robust multi-feature extraction.
  3. Guided Cortical Attention Feedback Module (GCAFM): This module implements a "predictive coding" mechanism, inspired by how the brain refines interpretations. It uses high-level semantic information ("diagnostic hypotheses") to iteratively refine low-level feature analysis, effectively integrating top-down feedback.

These modules are unified within a Polyp Encoder-Decoder Module (PEDM) that enforces spatial-semantic consistency through resolution-adaptive feedback, preventing attention drift and ensuring a coherent segmentation output. The entire system is trained end-to-end with a composite loss function that includes both segmentation accuracy (Dice loss) and bio-inspired components that enforce feedback consistency.

Datasets

GRAFNet was trained and evaluated on five public benchmarks:

  • Kvasir-SEG: A widely used dataset for polyp segmentation.
  • CVC-300: A dataset with precise masks.
  • CVC-ColonDB: A dataset featuring varied polyp types.
  • CVC-ClinicDB: A common benchmark for colonoscopy.
  • PolypGen: A large, multi-center collection of images and videos, used to test generalizability.

Key Insights and Ideas

  • Biologically inspired design pays off: The paper strongly suggests that mimicking the human visual system's hierarchical and feedback-driven processing leads to superior performance in medical image segmentation compared to purely feedforward, unidirectional models.
  • Feedback is crucial for refinement: The GCAFM module, which implements top-down feedback, was shown to be critical for improving segmentation accuracy and preventing attention drift across different scales.
  • Parallel pathways for rich features: The MSRM's replication of retinal pathways allows for a more comprehensive extraction of visual cues (texture, shape, color) than single-pathway approaches.
  • Orientation matters for subtle lesions: The GAAM's focus on orientation-tuned features proved particularly effective in detecting subtle, flat polyps that are easily missed by other methods.
  • Generalization through biological plausibility: The neurobiological design of GRAFNet appears to contribute to its strong performance across diverse datasets and imaging conditions, indicating better generalization capabilities.

Key Results

GRAFNet achieved state-of-the-art performance across multiple datasets, demonstrating significant improvements over existing methods:

  • Accuracy: GRAFNet achieved 3-8% higher Dice scores and 10-20% higher generalisation on unseen data compared to leading methods. For instance, on CVC-ClinicDB and Kvasir-SEG, it led in 10/12 metrics, with Dice scores of 0.9290 and 0.9146 respectively.
  • False Positive Reduction: The model significantly reduced false positives, particularly for normal anatomical structures like haustral folds, with a 19.1% FPR reduction compared to MDPNet.
  • Subtle Lesion Detection: GRAFNet showed notable gains in detecting flat (<3mm) and subtle (3-5mm) polyps, with improved Dice scores and sensitivity.
  • Efficiency: Despite its complex architecture, GRAFNet offered a good trade-off between efficiency and accuracy, being significantly faster than comparable models while maintaining high performance.

Limitations and Future Work

  • Computational Complexity: While GRAFNet achieves excellent performance, its computational complexity might still be a challenge for real-time clinical deployment.
  • Future Work: The authors suggest exploring lightweight adaptations through neural architecture search and pruning, integrating temporal modeling for video analysis, and extending the model for multi-task learning (e.g., joint detection, classification, and characterization). They also aim to further improve efficiency and broaden clinical utility.

Summary updated

BibTeX
@misc{fofanah2026grafnet,
  author = {Abdul Joseph Fofanah and Lian Wen and Alpha Alimamy Kamara and Zhongyi Zhang and David Chen and Albert Patrick Sankoh},
  title = {{GRAFNet: Multiscale Retinal Processing via Guided Cortical Attention Feedback for Enhancing Medical Image Polyp Segmentation}},
  year = {2026},
  eprint = {2602.15072},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2602.15072},
}
Figure