Abstract
Endoscopy is a crucial tool for diagnosing the gastrointestinal tract, but its effectiveness is often limited by a narrow field of view and the dynamic nature of the internal environment, especially in the esophagus, where complex and repetitive patterns make image stitching challenging. This paper introduces a novel automatic image unfolding and stitching framework tailored for esophageal videos captured during endoscopy. The method combines feature matching algorithms, including LoFTR, SIFT, and ORB, to create a feature filtering pool and employs a Density-Weighted Homography Optimization (DWHO) algorithm to enhance stitching accuracy. By merging consecutive frames, the framework generates a detailed panoramic view of the esophagus, enabling thorough and accurate visual analysis. Experimental results show the framework achieves low Root Mean Square Error (RMSE) and high Structural Similarity Index (SSIM) across extensive video sequences, demonstrating its potential for clinical use and improving the quality and continuity of endoscopic visual data.
1. Introduction
Endoscopy is the gold standard for diagnosing gastrointestinal issues, yet it faces a significant "keyhole" problem. The camera’s field of view is extremely narrow, and the tubular, dynamic structure of the esophagus results in a series of distorted, circular video frames. For clinicians, maintaining spatial context is difficult, and repetitive tissue patterns make it challenging to ensure comprehensive coverage. Traditional image stitching techniques often fail in this environment due to tissue deformation, camera instability, and the lack of distinct geometric landmarks in medical textures.
2. Methodology
The proposed framework addresses these challenges through a three-step pipeline designed to transform circular "peephole" video into a flat, high-resolution panoramic map.
2.1 Image Unfolding
The system utilizes a pre-trained AI model, Depth Anything, to generate a depth map for each video frame. By identifying the center of the esophageal tube—the point furthest from the camera—the framework mathematically "unwraps" the circular image into a flat, rectangular strip using a Virtual Cylindrical Grid. This step provides a consistent geometric reference for subsequent processing.
2.2 Hybrid Feature Matching
To align consecutive frames, the researchers implemented a "feature filtering pool" that combines traditional computer vision techniques (SIFT and ORB) with a modern deep-learning matcher (LoFTR). This hybrid approach ensures robustness; if one method fails to find landmarks on the slippery, low-texture esophageal wall, the others compensate to maintain alignment.
2.3 Density-Weighted Homography Optimization (DWHO)
The core mathematical contribution of this work is the DWHO algorithm. Standard stitching can be skewed if feature points cluster in a single region of the image. DWHO assigns weights to feature points based on their spatial density, ensuring a balanced alignment that prevents warping and preserves the anatomical integrity of the reconstructed panorama.
3. Experimental Evaluation
3.1 Datasets and Metrics
The framework was evaluated using the VUMC Dataset, which includes five endoscopy videos from pediatric patients diagnosed with Eosinophilic Esophagitis (EoE). A set of 300 frames was extracted for rigorous testing against standard stitching techniques using Structural Similarity Index (SSIM) and Root Mean Square Error (RMSE).
3.2 Results
The DWHO framework outperformed standard matching methods across both quantitative and qualitative metrics.
- Quantitative Performance: The framework achieved a superior SSIM of 0.502, outperforming LoFTR (0.492), SIFT (0.461), and ORB (0.454). It also recorded the lowest RMSE (5.776), indicating the highest alignment accuracy among the tested methods.
- Qualitative Performance: Visual results demonstrate that while traditional methods often produce jagged or fragmented panoramas, the DWHO method generates smooth, continuous strips of the esophageal lining, providing a clearer diagnostic view.
4. Conclusion and Future Work
This framework demonstrates the potential of combining AI-driven depth estimation with density-weighted optimization to overcome the inherent limitations of endoscopic imaging. By providing a stabilized, panoramic view, the system could significantly reduce the risk of missing critical pathologies. Future research will focus on expanding the clinical dataset and optimizing the pipeline for real-time processing during live medical procedures.
Summary updated