Ophtimus-V2-Tx: A Compact Domain-Specific LLM for Ophthalmic Diagnosis and Treatment Planning

Minwook Kwon, Kuk Jin Jang, Seung Ju Baek, Yong Seop Han, Hyon‐Young Choi, Insup Lee, Jin Hyun KimResearch Square 2025 · DOI

Abstract

Recent advances in Large language models (LLMs) have shown promising results in clinical decision support, but they often fall short when applied to case-specific reasoning required in real-world medical practice. We present Ophtimus-V2-Tx, a lightweight, domain-specific Small LM (SLM) fine-tuned on over 10,000 ophthalmology case reports, built upon an 8B-parameter base model. The model is optimized to generate diagnosis and treatment suggestions aligned with standardized coding systems such as ICD-10-CM, ATC, and ICD-10-PCS. Using the CliBench framework, we evaluate its performance across four clinical tasks: primary diagnosis, secondary diagnosis, medication recommendation, and surgical procedure prediction. Ophtimus-V2-Tx achieved the highest full-code F1 score in medication (0.32) and surgical tasks (0.16), and outperformed GPT-4o in primary diagnosis (F1: 0.28 vs. 0.25). It also showed strong topic-specific performance, attaining 0.80 accuracy on multiple-choice questions related to Uveitis, a clinically complex condition. Our contributions include: (1) empirical validation of case-based fine-tuning for clinical task adaptation, (2) application of a hierarchical benchmarking framework for multi-dimensional evaluation, and (3) a reproducible pipeline for building efficient, deployable medical LLMs. These findings demonstrate the feasibility of compact, domain-adapted models in delivering competitive clinical performance.

Introduction

While general-purpose Large Language Models (LLMs) like GPT-4o demonstrate impressive broad knowledge, they often struggle with the "last mile" of clinical practice. Medical specialties, particularly ophthalmology, require sophisticated case-specific reasoning—the ability to synthesize patient narratives, including chief complaints, medical history, and examination findings, into precise diagnostic and therapeutic plans. Generalist models, typically trained on broad biomedical corpora like Wikipedia, excel at defining diseases but often falter when navigating the nuanced complexities of actual patient care. Furthermore, the high computational demands of frontier models pose significant barriers to deployment in privacy-sensitive and resource-constrained clinical environments.

To address these challenges, we developed Ophtimus-V2-Tx, a compact 8B-parameter model specialized for ophthalmology. By prioritizing domain-specific depth over general-purpose scale, Ophtimus-V2-Tx demonstrates that smaller, targeted models can outperform generalist giants in critical clinical tasks.

Methodology

The development of Ophtimus-V2-Tx followed a multi-stage pipeline designed for clinical depth and computational efficiency:

  1. Foundation and Pre-training: We utilized LLaMA 3.1-8B as the base model. It was first adapted into Ophtimus-V2-Base through domain-specific pre-training on a massive ophthalmology corpus comprising 18.4 million PubMed articles and 4 million words from specialized textbooks.
  2. Case-Based Fine-Tuning: The core innovation lies in fine-tuning the model on 9,269 structured case reports. Unlike standard Q&A pairs, these reports provide end-to-end clinical narratives, teaching the model the workflow of expert medical thinking—from initial complaint to follow-up plans.
  3. Efficiency and Reproducibility: We employed LoRA (Low-Rank Adaptation) to fine-tune a small subset of parameters, ensuring the process remains lightweight and deployable.
  4. Evaluation with CliBench: Performance was measured using CliBench, which maps natural language outputs to standardized medical codes (ICD-10-CM, ATC, and ICD-10-PCS). This allows for a hierarchical evaluation of accuracy across different levels of diagnostic specificity.

Results and Analysis

Ophtimus-V2-Tx consistently outperformed or matched much larger models across several key clinical benchmarks.

Clinical Task Performance

In primary diagnosis, Ophtimus-V2-Tx achieved a full-code F1 score of 0.28, surpassing GPT-4o’s 0.25. It also led in surgical prediction (F1: 0.16 vs. 0.15) and matched GPT-4o in medication recommendations (F1: 0.32). Even when the model missed the exact "full-code" diagnosis, its predictions remained clinically relevant, showing high accuracy at broader levels of the ICD taxonomy.

Subdomain Mastery and Linguistic Alignment

The model demonstrated exceptional performance in specialized subfields. For instance, in the complex area of Uveitis, it achieved 0.80 accuracy on multiple-choice tasks.

Model accuracy across various ophthalmological topics, highlighting the performance of Ophtimus-V2-Tx compared to generalist models.

Furthermore, Ophtimus-V2-Tx showed superior linguistic alignment with human clinical notes. It achieved a ROUGE-L score of 0.40, significantly higher than GPT-4o’s 0.17, indicating that its generated narratives more closely mirror professional medical documentation.

Linguistic alignment metrics (ROUGE-L, BLEU, METEOR, SEMSCORE) comparing Ophtimus-V2-Tx against other models.

Specialized Specialization Trade-offs

Specialization comes with trade-offs. The treatment-optimized version (Ophtimus-V2-Tx) performed slightly lower on general multiple-choice tests compared to its instruction-tuned counterpart (Ophtimus-V2-Inst). This suggests that focusing on clinical reasoning may slightly dilute the model's ability to recall isolated, textbook facts.

Conclusion and Future Work

Ophtimus-V2-Tx proves that domain-adapted Small Language Models can deliver expert-level performance in specialized medical fields. By training on clinical narratives rather than just facts, we can create models that better understand the diagnostic workflow.

Future work will focus on:

  • Multimodality: Integrating visual data such as fundus photography and OCT scans to move beyond text-only reasoning.
  • Ensemble Architectures: Exploring agent-based systems where multiple specialized small models collaborate to handle broader medical reasoning.
  • Data Accessibility: Addressing the challenges of restricted medical datasets to improve the reproducibility of clinical AI research.

Summary updated

Abstract

<title>Abstract</title> Recent advances in Large language models (LLMs) have shown promising results in clinical decision support, but they often fall short when applied to case-specific reasoning required in real-world medical practice. We present Ophtimus-V2-Tx, a lightweight, domain-specific Small LM (SLM) fine-tuned on over 10,000 ophthalmology case reports, built upon an 8B-parameter base model. The model is optimized to generate diagnosis and treatment suggestions aligned with standardized coding systems such as ICD-10-CM, ATC, and ICD-10-PCS. Using the CliBench framework, we evaluate its performance across four clinical tasks: primary diagnosis, secondary diagnosis, medication recommendation, and surgical procedure prediction. Ophtimus-V2- Tx achieved the highest full-code F1 score in medication (0.32) and surgical tasks (0.16), and outperformed GPT-4o in primary diagnosis (F1: 0.28 vs. 0.25). It also showed strong topic-specific performance, attaining 0.80 accuracy on multiple-choice questions related to Uveitis, a clinically complex condition. Our contributions include: (1) empirical validation of case-based fine-tuning for clinical task adaptation, (2) application of a hierarchical benchmarking framework for multi-dimensional evaluation, and (3) a reproducible pipeline for building efficient, deployable medical LLMs. These findings demonstrate the feasibility of compact, domain-adapted models in delivering competitive clinical performance.
BibTeX
@inproceedings{kwon2025ophtimus,
  author = {Minwook Kwon and Kuk Jin Jang and Seung Ju Baek and Yong Seop Han and Hyon‐Young Choi and Insup Lee and Jin Hyun Kim},
  title = {{Ophtimus-V2-Tx: A Compact Domain-Specific LLM for Ophthalmic Diagnosis and Treatment Planning}},
  booktitle = {Research Square},
  year = {2025},
  doi = {10.21203/rs.3.rs-7191523/v1},
}
Figure