TL;DR
Researchers have developed EndoComparer, a vision-language model (VLM) designed to "spot the difference" between two surgical videos. By using a specialized visual adapter for each video and a "perceive-then-compare" reasoning strategy, the model significantly outperforms general-purpose AI in identifying variations in surgical skill, tool usage, and anatomy.
What Problem Does This Paper Address?
In surgical training, learning often happens through comparison: “Why was the dissection in Video A cleaner than in Video B?” While Vision-Language Models (VLMs) have become adept at describing single videos, they struggle with comparative reasoning.
The core challenge is the "binding problem." When a model looks at two videos of the same procedure (e.g., a gallbladder removal), it often gets confused about which observation belongs to which video. Because the organs, tools, and background look nearly identical across different cases, standard models frequently attribute a specific action to the wrong video slot. This makes them unreliable for providing the nuanced feedback required for surgical education.
Methods
The authors introduce two novel components to help the model keep its "eyes" and "thoughts" organized:
- Pseudo-Siamese Visual LoRA (PSVL): Standard models use a "Siamese" approach, applying the exact same weights to both videos. EndoComparer uses a shared, frozen vision encoder but adds a dedicated Low-Rank Adapter (LoRA) for each video slot. This creates separate visual pathways for "Video 1" and "Video 2," helping the model maintain the identity of each stream from the very first layer of processing without doubling the computational cost.
- Perceive-then-Compare Chain of Thought (PtC-CoT): Instead of jumping straight to an answer, the model is trained to follow a specific reasoning path. It must first state what is happening in Video 1, then state what is happening in Video 2, and only then derive the comparative answer. This "thinking out loud" ensures the final judgment is grounded in distinct observations for each video.
Datasets
The researchers curated EndoDiffVQA, a new benchmark for comparative surgical VQA.
- Scale: Nearly 11,000 multiple-choice questions (four options each).
- Source Material: Derived from 1,762 clips of laparoscopic cholecystectomy (gallbladder removal) and Roux-en-Y gastric bypass.
- Categories: The questions cover four critical domains: Anatomy (visibility, color, state), Tools (presence, count, engagement), Actions (interactions, goals), and Skill (proficiency scores).
- Distractors: Unlike random multiple-choice options, the "wrong" answers are designed to mimic common model failures, such as swapping the facts between the two videos.
Key Insights and Ideas
- Attribution is Harder than Perception: The study found that models often "see" the correct facts but "attach" them to the wrong video. This confirms that the bottleneck in comparative VQA isn't just visual recognition, but the logical binding of facts to sources.
- The "Video-Swap" Litmus Test: To prove their model wasn't just guessing, the authors performed a swap control—exchanging the order of Video 1 and Video 2. While standard fine-tuned VLMs saw their accuracy plummet (losing 5–8 points), EndoComparer remained robust, losing only 0.6 points. This proves the model truly understands which video is which.
- Skill is the Ultimate Challenge: Comparing surgical skill is much harder than identifying a tool. The gains from EndoComparer were most pronounced in the "Skill" category, where the model must integrate observations over minutes of footage rather than just identifying a single frame.
Key Results
- State-of-the-Art Performance: EndoComparer achieved a 54.0 Strict score, outperforming its fine-tuned backbone (Qwen3.5-9B) by 5.7 points.
- Beating Proprietary Giants: It surpassed Gemini 3.6 Flash by 7.1 points, despite Gemini being a much larger, proprietary model.
- Synergy: Ablation studies showed that PSVL (the visual fix) and PtC-CoT (the reasoning fix) are complementary; using both together yielded significantly better results than either alone, especially for skill assessment.
Limitations and Future Work
The authors acknowledge several areas for growth:
- Two-Video Limit: The current framework is limited to comparing exactly two videos at a time.
- Multiple Choice Constraints: The benchmark uses a multiple-choice format, which measures if a model can select the right relation but doesn't test its ability to describe differences in free-form natural language.
- Noise Inheritance: The dataset is built on existing captions and expert scores, meaning it inherits any noise or inter-rater variability present in those source materials.
- Clinical Utility: While a strong step toward AI coaching, the model currently acts more as a "triage" tool for ranking cases rather than a source of open-ended, unsupervised surgical feedback.
Summary updated