Enhanced Feature-based Image Stitching for Endoscopic Videos in Pediatric Eosinophilic Esophagitis

Juming Xiong, Muyang Li, Ruining Deng, Tianyuan Yao, Shunxing Bao, Regina N Tyree, Girish Hiremath, Yuankai Huo

TL;DR

Traditional image stitching algorithms fail on esophageal endoscopic videos due to smooth tissue textures and unpredictable camera rotation. This paper introduces a specialized four-step preprocessing pipeline—incorporating keyframe selection, ellipse-based rotation correction, polar unwrapping, and contrast enhancement—that drastically improves feature matching to enable seamless panoramic visualization.

What Problem Does This Paper Address?

Esophageal endoscopy is a vital tool for diagnosing and monitoring gastrointestinal disorders like Eosinophilic Esophagitis (EoE). However, reviewing these video recordings is a tedious task. Clinicians must constantly reorient themselves and mentally piece together fragmented views to get a comprehensive understanding of the tissue.

Automatically stitching these video frames into a continuous, panoramic 2D image would save time and improve diagnostic accuracy. Unfortunately, standard image stitching algorithms (like the ones used to create panoramas on your smartphone) fail when applied to the esophagus. Esophageal images suffer from three unique challenges:

  1. Smooth textures: The esophageal lining lacks the sharp, distinct feature points required by traditional matching algorithms.
  2. Non-horizontal orientation: The camera constantly shifts and rotates as it navigates the tubular organ.
  3. Geometric distortion: Capturing a 3D tube with a forward-facing camera distorts the spatial relationships of the tissue surface.
Figure 1. The upper panel shows the general image stitching result and it can be successfully stitched. The lower panel shows the failed endoscopic image stitching result by using the traditional feature-based image stitching method.

Methods

To overcome these challenges, the authors developed a novel preprocessing pipeline that prepares endoscopic video frames for standard feature-based stitching tools. The pipeline consists of four main steps:

  1. Keyframe Selection: To eliminate motion blur and common occlusions, the algorithm automatically discards the first and last 3 seconds of each video. It then applies uniform sampling, extracting one frame every 5 frames.
  2. Image Rotation Adjustment: To counteract the uncontrollable spinning of the endoscopy camera, the pipeline identifies the "deep area" (the dark center of the esophageal lumen) using a pixel-value threshold. It fits an ellipse to this contour and uses its angle to rotate the image back to a standardized orientation.
  3. Surface Unwrapping: To convert the tubular perspective into a flat 2D surface, the algorithm defines an annular (ring-shaped) region between two concentric circles centered at the deepest point of the lumen. It then uses a polar coordinate transformation to "unwrap" this ring into a flat rectangular grid.
  4. Feature Point Matching Enhancement: Because the unwrapped mucosal tissue is still relatively featureless, the authors apply Adaptive Histogram Equalization (AHE) to boost local contrast. Finally, the Scale-Invariant Feature Transform (SIFT) detects and matches keypoints, which are filtered using a ratio test and refined via Random Sample Consensus (RANSAC) to eliminate outliers.

Datasets

The pipeline was validated on a clinical dataset consisting of:

  • 20 endoscopy videos containing a total of 35,652 frames.
  • Data collected from 5 pediatric patients (aged 6–18) diagnosed with EoE at the Monroe Carrell Jr. Children's Hospital at Vanderbilt University Medical Center (VUMC).

Key Insights and Ideas

  • The Lumen as an Anchor: Using the darkest region of the image to fit an ellipse provides an elegant, domain-specific way to calculate camera rotation without needing external spatial sensors.
  • Geometry Transformation is Essential: Standard stitching assumes a planar or spherical camera sweep across a scene. Unwrapping the cylindrical geometry of the esophagus into a flat strip fundamentally changes the problem space, making it compatible with traditional 2D stitching frameworks.
  • Amplifying Subtle Textures: While standard SIFT fails on smooth tissue, pairing it with local contrast enhancement (AHE) uncovers micro-textures in the esophageal lining that are otherwise invisible to feature detectors.

Key Results

Because clinical endoscopy lacks an absolute "ground truth" panorama, the authors evaluated stitching quality by measuring the number of valid feature point match pairs between overlapping frames.

The experiments compared three configurations:

  • Original Unwrapped: Yielded only 5 valid feature points, completely failing to stitch.
  • AHE Unwrapped (No Rotation Correction): Improved to 52 valid feature points, but the resulting panoramas exhibited large, misaligned gaps.
  • AHE Rotated Unwrapped (Full Pipeline): Achieved the highest performance with 57 valid feature points, delivering the best alignment and most seamless panoramic results.

Limitations and Future Work

  • Indirect Evaluation: The reliance on valid feature match counts is an indirect metric; the paper does not feature a direct quantitative evaluation against a physical ground truth phantom.
  • Small Cohort Size: The dataset is limited to 20 videos from just 5 pediatric patients. Testing on a larger, more diverse patient population is required to confirm the pipeline's generalizability.
  • Downstream Blending: The paper focuses primarily on the preprocessing and matching stages. Fully automating global bundle adjustment and seamless image blending for long, continuous video sequences remains an open engineering challenge.

Summary updated

Abstract

Video endoscopy represents a major advance in the investigation of gastrointestinal diseases. Reviewing endoscopy videos often involves frequent adjustments and reorientations to piece together a complete view, which can be both time-consuming and prone to errors. Image stitching techniques address this issue by providing a continuous and complete visualization of the examined area. However, endoscopic images, particularly those of the esophagus, present unique challenges. The smooth surface, lack of distinct feature points, and non-horizontal orientation complicate the stitching process, rendering traditional feature-based methods often ineffective for these types of images. In this paper, we propose a novel preprocessing pipeline designed to enhance endoscopic image stitching through advanced computational techniques. Our approach converts endoscopic video data into continuous 2D images by following four key steps: (1) keyframe selection, (2) image rotation adjustment to correct distortions, (3) surface unwrapping using polar coordinate transformation to generate a flat image, and (4) feature point matching enhanced by Adaptive Histogram Equalization for improved feature detection. We evaluate stitching quality through the assessment of valid feature point match pairs. Experiments conducted on 20 pediatric endoscopy videos demonstrate that our method significantly improves image alignment and stitching quality compared to traditional techniques, laying a robust foundation for more effective panoramic image creation.
BibTeX
@misc{xiong2025enhanced,
  author = {Juming Xiong and Muyang Li and Ruining Deng and Tianyuan Yao and Shunxing Bao and Regina N Tyree and Girish Hiremath and Yuankai Huo},
  title = {{Enhanced Feature-based Image Stitching for Endoscopic Videos in Pediatric Eosinophilic Esophagitis}},
  year = {2025},
  eprint = {2502.04207},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.04207},
}
Figure