Ophora: A Large-Scale Data-Driven Text-Guided Ophthalmic Surgical Video Generation Model

Wei Li, Ming Hu, Guoan Wang, Lihao Liu, Kaijing Zhou, Junzhi Ning, Xin Guo, Zongyuan Ge, Lixu Gu, Junjun He

Introduction

The development of AI-assisted surgical systems is often bottlenecked by the scarcity of high-quality, annotated video data. Ophthalmic surgery, characterized by intricate maneuvers and delicate anatomical structures, presents unique challenges for data collection due to strict privacy regulations and the high cost of expert labeling. While text-to-video (T2V) generation offers a potential solution for synthesizing training data, existing models often struggle with fine-grained surgical details, temporal consistency, and the presence of visual noise like watermarks or subtitles.

To address these challenges, we present Ophora, a pioneering T2V model specifically designed for ophthalmic surgery. By leveraging a massive new dataset and a progressive tuning strategy, Ophora generates realistic, temporally consistent, and privacy-preserved surgical videos that can significantly improve the training of downstream surgical AI systems.

Methodology

Ophora's development involves a sophisticated data curation pipeline and a multi-stage training scheme designed to bridge the gap between general-purpose video generation and the specialized needs of the operating room.

Data Curation and Refinement

The foundation of Ophora is the Ophora-160K dataset, consisting of 162,185 video-instruction pairs derived from nearly 10,000 narrative surgical videos. To bridge the gap between messy human narration and structured machine instructions, the authors utilized a Large Language Model (Qwen2.5-72B) to strip away irrelevant educational content and convert descriptions into precise generation instructions. Additionally, dynamics-based filtering was applied to remove clips with excessive camera shake or static content, ensuring the model learns from high-quality motion.

Model Architecture

Ophora is built upon the CogVideoX-2b backbone, a latent diffusion model. It incorporates a 3D Variational Autoencoder (VAE) and a transformer-based denoising network. Unlike previous surgical T2V attempts that added temporal layers to static image models, Ophora uses a native 3D video architecture. This results in much smoother, more realistic instrument movements and spatial-temporal coherence.

Progressive Video-Instruction Tuning

The model undergoes a two-stage tuning process to ensure both domain expertise and visual cleanliness:

  1. Transfer Pre-Training (TPT): The model is first trained on the full 160K dataset to soak up general ophthalmic domain knowledge and surgical semantics.
  2. Privacy-Preserving Fine-Tuning ($P^2FT$): To ensure the model doesn't generate "dirty" videos with text overlays, a Vision-Language Model (Qwen2.5-VL) was used to identify a "clean" subset of 28,000 clips (Ophora-28K). Fine-tuning on this subset ensures the output resembles a raw surgical feed rather than an edited educational video.

Results and Evaluation

Ophora was evaluated against state-of-the-art surgical video models, including Endora and Bora, across several key metrics.

Generative Quality

Ophora achieved a Fréchet Inception Distance (FID) of 33.72, indicating superior realism. Furthermore, its Fréchet Video Distance (FVD) of 276.96 was nearly half that of previous models, demonstrating significantly improved temporal quality and motion smoothness. This suggests that the 3D architecture and refined instructions effectively capture the nuances of surgical dynamics.

Downstream Utility

A critical test for synthetic data is its ability to improve real-world AI performance. When synthetic Ophora videos were integrated into the training set for a surgical workflow classifier (MViTv2) using the OphNet benchmark, the Top-1 accuracy increased from 37.92% to 42.24%. This confirms that Ophora acts as a "force multiplier," augmenting scarce real-world data effectively.

Expert Validation

In blind evaluations, professional ophthalmologists rated Ophora highly for "Action Realism" and "Phase Matching," confirming that the generated videos are medically plausible and align with specific surgical phases (validated against the Cataract-1K dataset).

Conclusion and Future Work

Ophora represents a significant step forward in surgical video synthesis. By combining LLM-driven data refinement with a native 3D video architecture, it overcomes the limitations of general-purpose T2V models in the medical domain. While currently limited to short clips (averaging ~5.5 seconds) and specialized for ophthalmology, the framework provides a blueprint for other surgical specialties. Future directions include exploring longer-duration video generation and expanding the model's scope to broader surgical contexts.

Summary updated

BibTeX
@inproceedings{li2025ophora,
  author = {Wei Li and Ming Hu and Guoan Wang and Lihao Liu and Kaijing Zhou and Junzhi Ning and Xin Guo and Zongyuan Ge and Lixu Gu and Junjun He},
  title = {{Ophora: A Large-Scale Data-Driven Text-Guided Ophthalmic Surgical Video Generation Model}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2025},
  doi = {10.1007/978-3-032-05114-1\_41},
}
Figure