Abstract
Video endoscopy represents a major advance in the investigation of gastrointestinal diseases. However, reviewing endoscopy videos often involves frequent adjustments and reorientations to piece together a complete view, which can be both time-consuming and prone to errors. Image stitching techniques address this issue by providing a continuous and complete visualization of the examined area. Endoscopic images, particularly those of the esophagus, present unique challenges: smooth surfaces, a lack of distinct feature points, and non-horizontal orientation render traditional feature-based methods often ineffective. In this paper, we propose a novel preprocessing pipeline designed to enhance endoscopic image stitching. Our approach converts endoscopic video data into continuous 2D images through keyframe selection, image rotation adjustment, surface unwrapping using polar coordinate transformation, and feature point matching enhanced by Adaptive Histogram Equalization. Experiments conducted on 20 pediatric endoscopy videos demonstrate that our method significantly improves image alignment and stitching quality compared to traditional techniques.
Introduction
Reviewing endoscopy videos is a grueling task for clinicians, requiring them to mentally reconstruct a 3D organ from fragmented, rotating, and often blurry 2D video frames. While "image stitching"—the process of combining multiple images into a single panoramic view—is a standard solution in landscape photography, it frequently fails in the context of the esophagus.
The esophagus presents a "perfect storm" of computer vision challenges. The mucosal lining is smooth and uniform, offering very few distinct landmarks for algorithms to track. Additionally, as the endoscope moves, it spins unpredictably, and the tubular geometry creates perspective-heavy circular views that standard stitching algorithms are not designed to handle.
Methodology
To address these challenges, we propose a four-stage preprocessing pipeline to prepare endoscopic data for standard stitching techniques:
- Keyframe Selection: To handle motion blur and occlusions, the system discards the first and last three seconds of video and uses uniform sampling (extracting one frame every five) to reduce redundancy.
- Image Rotation Adjustment: The algorithm identifies the "deep area" (the lumen) using a pixel threshold and fits an ellipse to this area. The ellipse's angle is used to rotate every frame to a standardized orientation, "zeroing out" the camera's spinning motion.
- Surface Unwrapping: Using polar coordinate transformation, the circular "donut" of the esophageal wall is "unwrapped" into a rectangular 2D strip. This flattens the geometry, making it look more like a traditional landscape.
- Feature Enhancement & Matching: We apply Adaptive Histogram Equalization (AHE) to boost local contrast, revealing subtle micro-textures in the smooth tissue. We then use SIFT (Scale-Invariant Feature Transform) to find matching points and RANSAC to filter out false matches.
Results and Key Insights
The study utilized a clinical dataset of 20 pediatric endoscopy videos from five children diagnosed with Eosinophilic Esophagitis (EoE). We compared three versions of our pipeline: Original (unwrapped only), AHE (unwrapped + contrast boost), and AHE Rotated (the full proposed pipeline).
The "AHE Rotated" method yielded the highest number of valid feature matches. In a representative test case, the original image provided only 5 valid matches (leading to stitching failure), while the full pipeline achieved 57 valid matches. Using the Wilcoxon signed-rank test, we confirmed that the improvements in feature detection were statistically significant ($p < 0.05$) across all 20 videos. The final stitched panoramas showed significantly better alignment and fewer artifacts compared to traditional methods.
Conclusion
This research demonstrates that geometry matters more than pixels when dealing with tubular organs. By mathematically "unwrapping" the esophagus into a flat plane and using the lumen as a rotational anchor, we can make the problem solvable for standard algorithms. While current work uses feature matches as a proxy for quality, future research will focus on real-time processing and 3D reconstruction to provide even more intuitive visualizations for clinicians.
Summary updated