Retrieving challenging vessel connections in retinal images by line co-occurrence statistics

Samaneh Abbasi-Sureshjani, Jiong Zhang, Remco Duits, Bart M. ter Haar Romeny

TL;DR

Researchers have discovered that the statistical "logic" of how blood vessels are laid out in the human retina almost perfectly matches mathematical models of the human visual cortex. By leveraging these "line co-occurrence statistics," they developed a method to automatically reconnect interrupted or crossing vessel segments in medical images with high precision.

What Problem Does This Paper Address?

Mapping the retinal vasculature is a cornerstone of diagnosing diseases like hypertension and diabetic retinopathy. However, even the best automated segmentation algorithms struggle with "broken" data. Vessels often appear disconnected due to low image contrast, or they become ambiguous where they branch (bifurcations) or cross over one another.

While humans can easily "see" which segments belong together—thanks to our brain's innate ability to perceive continuous contours (Gestalt laws)—computers traditionally rely on manually designed cost functions that often fail in complex scenarios. This paper addresses the challenge of automated perceptual grouping: how can we teach a computer to "reason" about vessel connectivity as effectively as a human expert?

Methods

The authors bridge the gap between biological vision and computer vision using a multi-step framework:

  1. Orientation Scores: They "lift" 2D retinal images into a 3D space ($R^2 \times P^1$) that includes position and orientation. This allows the algorithm to distinguish between vessels crossing at different angles at the same pixel location.
  2. Line Co-occurrence Statistics: Instead of manually defining how vessels should connect, the authors looked at how they actually connect across thousands of points in real datasets. They calculated the probability that two vessel segments at specific relative positions and orientations belong to the same structure.
  3. The Cortical Connection: They compared these data-driven statistics to Mumford’s direction process, a stochastic model used to describe how neurons in the primary visual cortex (V1) represent contour completion.
  4. Spectral Clustering: Finally, they used the learned statistical kernel to build an "affinity matrix." By combining geometric probability with vessel intensity similarity, they used self-tuning spectral clustering to group disconnected pixels into individual, continuous vessels.

Datasets

The study utilized two primary benchmarks in the field:

  • DRIVE Dataset: 40 color fundus images (565x584 resolution).
  • IOSTAR Dataset: 24 images taken with a Scanning Laser Ophthalmoscope (SLO), which provides higher resolution (1024x1024) and different contrast characteristics.

The authors used the "ground truth" manual segmentations from these datasets to train and validate their statistical models.

Key Insights and Ideas

  • Nature Mirrors Math: The most striking finding is that the statistical distribution of vessel connections in the retina matches the theoretical model of the human visual cortex with less than 2% error. This suggests that the "continuation logic" our brains use to see the world is remarkably well-suited for medical imaging tasks.
  • Dataset Independence: The statistical kernels derived from the DRIVE dataset were nearly identical to those from IOSTAR. This is a "big win" for robustness; it means the geometric rules of vessel growth are universal enough that a model trained on one type of camera can likely work on another.
  • Projective Line Bundles: By modeling the connectivity on a "projective line bundle" (where orientation is $\pi$-periodic), the authors accounted for the fact that a vessel segment doesn't have a "head" or a "tail"—it’s just a line. This symmetrization was key to achieving the high match rate with biological models.

Key Results

  • Quantitative Accuracy: The best approximation of the statistical model using the symmetrized probabilistic model achieved a least-square error of 0.33% to 1.04% across different datasets.
  • Qualitative Success: The method was tested on "challenging" image patches containing bifurcations, crossings, and deliberate disconnections. In almost all cases, the spectral clustering correctly grouped pixels into their respective individual vessels, even when intensity information was missing.
  • Automation: The system is "self-tuning," meaning it doesn't require researchers to manually set thresholds for how many vessels are in a patch; the algorithm determines the number of salient groups automatically.

Limitations and Future Work

While the geometric model is powerful, it is not infallible. The authors note that the method can still struggle if:

  1. Extreme Noise: The vessel intensity or orientation changes too abruptly or is obscured by heavy noise.
  2. Complex Curvature: The current model focuses on orientation and position. The authors suggest that incorporating curvature and scale (vessel width) into the statistics could make the grouping even more robust.

Future work aims to extend this "line co-occurrence" approach to other curvilinear structures, such as corneal nerve fibers, plant roots, or even road networks in satellite imagery.

Summary updated

Abstract

Natural images contain often curvilinear structures, which might be disconnected, or partly occluded. Recovering the missing connection of disconnected structures is an open issue and needs appropriate geometric reasoning. We propose to find line co-occurrence statistics from the centerlines of blood vessels in retinal images and show its remarkable similarity to a well-known probabilistic model for the connectivity pattern in the primary visual cortex. Furthermore, the probabilistic model is trained from the data via statistics and used for automated grouping of interrupted vessels in a spectral clustering based approach. Several challenging image patches are investigated around junction points, where successful results indicate the perfect match of the trained model to the profiles of blood vessels in retinal images. Also, comparisons among several statistical models obtained from different datasets reveal their high similarity, i.e., they are independent of the dataset. On top of that the best approximation of the statistical model with the symmetrized extension of the probabilistic model on the projective line bundle is found with a least square error smaller than \(2\%\). Apparently, the direction process on the projective line bundle is a good continuation model for vessels in retinal images.
BibTeX
@inproceedings{abbasisureshjani2017retrieving,
  author = {Samaneh Abbasi-Sureshjani and Jiong Zhang and Remco Duits and Bart M. ter Haar Romeny},
  title = {{Retrieving challenging vessel connections in retinal images by line co-occurrence statistics}},
  booktitle = {Biol. Cybern},
  year = {2017},
  url = {https://openreview.net/forum?id=xFGWlUedYa},
}

Loading related content...

Figure