Introduction
In clinical settings, a physician's intent to order a medical intervention—such as a lab test or imaging—is often embedded within complex doctor-patient dialogues. Traditional systems rely on Large Language Models (LLMs) to rewrite these ambient conversations into structured search queries. However, this "intermediate rewriter" approach introduces significant latency, instability due to speech-to-text errors, and reduced transparency.
JEDA (Joint Embedding for Direct and Ambient clinical orders) addresses these challenges by employing a bi-encoder architecture that retrieves clinical orders directly from raw transcript chunks. By bypassing the LLM rewriter at inference time, JEDA achieves the millisecond latency required for real-time clinical assistants while significantly improving retrieval accuracy.
Methodology
JEDA is built upon a PubMedBERT backbone and optimized through a specialized training pipeline designed to handle the nuances of medical dialogue.
LLM-Guided Data Augmentation
The primary innovation in JEDA's training is the use of an offline LLM to generate "reasoning-enriched" training pairs. For each historical signed order, the LLM identifies relevant transcript segments and synthesizes four query variants: direct commands, ambient context, fused command-context, and clinical reasoning (the "why" behind the order). This multi-faceted supervision allows the model to learn the underlying clinical logic rather than relying on simple keyword matching.
Duplicate-Safe Contrastive Learning
To prevent the model from being penalized for retrieving semantically identical but lexically different orders, the researchers implemented a "duplicate-safe" contrastive objective. This approach masks in-batch negatives that actually point to the same clinical order, ensuring a more accurate representation of the medical embedding space.
Results and Analysis
JEDA was evaluated on a dataset of ~1,400 unseen clinical encounter pairs. The model demonstrated a 3.5x improvement in Recall@1 (0.28 vs. 0.08) compared to the base PubMedBERT model. Furthermore, JEDA outperformed much larger general-purpose embedding models, such as the 7B-parameter Mistral-based models, proving that domain-specific fine-tuning can outweigh raw parameter scale in specialized fields.
Geometric Alignment
Analysis of the embedding space reveals that JEDA significantly improves the cosine similarity between various query types and their corresponding clinical orders. As shown in the comparison below, JEDA (right) maintains consistently higher similarity scores across all query variants—including the challenging "context-only" ambient dialogue—compared to the PubMedBERT baseline (left).
The inclusion of "reasoning" during training acted as a geometric regularizer, helping the model distinguish between clinically similar but distinct orders by expanding the margins between their respective clusters.
Conclusion
JEDA demonstrates that a "query-free" approach to clinical order retrieval is not only feasible but superior to traditional LLM-rewriting pipelines in terms of speed and accuracy. By leveraging LLM-generated reasoning during training, the model develops a robust understanding of clinical intent from ambient speech. Future work will focus on incorporating medical ontologies to further refine the model's ability to distinguish between highly similar medical concepts.
Summary updated