Bladder image stitching algorithm for navigation and referencing using a standard cystoscope

Ming Li, Nicole Varble, Sandeep Gurram, Dilara J. Long, Vladimir Valera, Nikhil Gopal, Ivane Bakhutashvili, Sheridan Reed, William F. Pritchard, John W. Karanian, Sheng Xu, Bradford J. WoodScientific 2024 · DOI

Abstract

To aid in the diagnosis, monitoring, and surveillance of bladder carcinoma, this study developed a vision-based algorithm that creates referenceable 2D bladder maps from standard cystoscopy videos without specialized hardware. By sequentially stitching image frames based on matching surface features (SURF and RANSAC), the system generates a comprehensive map and enables real-time localization during follow-up procedures. Evaluated in 2D models, 3D phantoms, swine models, and retrospective clinical data, the algorithm demonstrated high feature-matching reliability (93-99% of frames) and rapid localization (under 5 seconds). This tool offers a cost-effective method for standardized documentation and longitudinal tracking of bladder lesions.

Introduction: The Challenge of Longitudinal Bladder Surveillance

Bladder cancer is characterized by high recurrence rates, necessitating frequent and lifelong surveillance via cystoscopy. Currently, urologists document findings using subjective 2D hand-drawn diagrams or raw video files. These methods lack spatial context, making it difficult to precisely revisit specific lesion sites during subsequent procedures. While advanced 3D reconstruction and electromagnetic tracking systems exist, their high cost and requirement for specialized hardware limit their accessibility, particularly in low-resource settings. There is a clear need for a hardware-agnostic, software-based solution that leverages existing video feeds to create a spatial record of the bladder.

Methodology: Vision-Based Stitching and Localization

The proposed pipeline consists of two primary components: a stitching algorithm for map generation and a "revisit" algorithm for localization.

Feature Detection and Map Generation

The system utilizes Speeded-Up Robust Feature (SURF) to identify stable landmarks, such as blood vessel bifurcations, within each video frame. To establish spatial relationships, Random Sample Consensus (RANSAC) calculates the affine transformation between sequential frames, accounting for rotation, translation, and zoom.

Visual pipeline of the image stitching process, showing feature detection, matching, and final image alignment.

To mitigate "drift"—the accumulation of small errors over long sequences—the authors implemented a drift correction step. This involves identifying similar features between spatially adjacent but temporally distant frames, allowing the algorithm to "close the loop" and maintain global map accuracy.

The Revisit Logic

Once a reference map is established, the algorithm can localize the cystoscope during a follow-up exam. By comparing new video frames against the existing map, the system identifies matching features to determine the scope's current position and orientation in near real-time.

Flowchart detailing the 'revisit' logic used to localize new video frames within a previously generated bladder map.

Evaluation and Results

The algorithm was validated across four scenarios: a 2D benchtop model, a 3D anthropomorphic phantom, a preclinical swine model, and retrospective human clinical video.

Performance Metrics

  • Feature Sufficiency: Between 93% and 99% of video frames contained sufficient features (≥6 matches) for successful tracking, confirming that natural bladder vasculature provides an adequate "fingerprint."
  • Localization Speed: In the 3D phantom, the system localized the initial frame of a new sweep within 4.5 seconds, with subsequent frames processed in approximately 0.35 seconds.
  • Clinical Feasibility: The algorithm successfully processed archived human cystoscopy data, generating a high-resolution composite map (9837×8585 pixels) in 168 seconds.
  • Hardware Agnosticism: The system performed reliably across various devices, from low-cost disposable flexible scopes (360p) to high-definition rigid surgical scopes (720p).

Discussion and Future Directions

By shifting the complexity from hardware to software, this algorithm provides a scalable path for advanced surgical navigation. The resulting maps offer a "ground truth" that reduces inter-operator variability and improves the precision of longitudinal monitoring.

However, limitations remain. The bladder is a non-rigid organ; variations in fluid distension can warp the mucosal surface, potentially complicating feature matching. Future work will focus on incorporating elastic registration to account for these deformations and conducting prospective clinical trials to evaluate the tool's impact on active surgical workflows and patient outcomes.

Summary updated

Abstract

To aid in the diagnosis, monitoring, and surveillance of bladder carcinoma, this study aimed to develop and test an algorithm that creates a referenceable bladder map rendered from standard cystoscopy videos without the need for specialized equipment. A vision-based algorithm was developed to generate 2D bladder maps from individual video frames, by sequentially stitching image frames based on matching surface features, and subsequently localize and track frames during reevaluation. The algorithm was developed and calibrated in a 2D model and 3D anthropomorphic bladder phantom. The performance was evaluated in vivo in swine and with retrospective clinical cystoscopy video. Results showed that the algorithm was capable of capturing and stitching intravesical images with different sweeping patterns. Between 93% and 99% of frames had sufficient features for bladder map generation. Upon reevaluation, the cystoscope accurately localized a frame within 4.5 s. In swine, a virtual mucosal surface map was generated that matched the explant anatomy. A surface map could be generated based on archived patient cystoscopy images. This tool could aid recording and referencing pathologic findings and biopsy or treatment locations for subsequent procedures and may have utility in patients with metachronous bladder cancer and in low-resource settings.
BibTeX
@inproceedings{li2024bladder,
  author = {Ming Li and Nicole Varble and Sandeep Gurram and Dilara J. Long and Vladimir Valera and Nikhil Gopal and Ivane Bakhutashvili and Sheridan Reed and William F. Pritchard and John W. Karanian and Sheng Xu and Bradford J. Wood},
  title = {{Bladder image stitching algorithm for navigation and referencing using a standard cystoscope}},
  booktitle = {Scientific},
  year = {2024},
  doi = {10.1038/s41598-024-80284-7},
}
Figure