Introduction
Adapting large language models (LLMs) to specialized domains such as medicine or finance is a critical requirement for enterprise deployment. While general-purpose models like GPT-4 are powerful, smaller, domain-specific models are often preferred for their lower latency, reduced operational costs, and better data privacy. However, the performance of these smaller models depends heavily on high-quality, domain-specific fine-tuning data.
The primary bottleneck in this process is the "data labeling gap." High-fidelity labels from human experts (e.g., physicians or financial analysts) are prohibitively expensive and difficult to scale. Conversely, low-fidelity labels generated by LLMs are inexpensive but prone to hallucinations and lack the technical nuance required for high-stakes applications. This paper introduces Interactive Multi-Fidelity Learning (IMFL), a framework designed to bridge this gap by strategically combining sparse human supervision with automated LLM annotation.
The IMFL Framework
IMFL treats the annotation process as a multi-fidelity learning problem. It uses a small budget of high-fidelity human labels to "teach" an LLM annotator, which then generates a larger volume of "silver" labels for training the target model. The framework is built on three core methodological pillars:
1. Exploration-Exploitation Query (EEQ) Strategy
To maximize the utility of every human label, IMFL moves beyond simple random sampling or basic uncertainty sampling. It employs a two-stage selection process:
- Exploration: The system uses $k$-means clustering on the unannotated data pool to ensure that the selected samples represent the full breadth of the domain.
- Exploitation: Within these clusters, the system identifies samples with the highest predictive uncertainty—those the current model is most likely to get wrong. This dual approach ensures the model learns both the general structure of the data and the specific nuances of difficult edge cases.
2. Similarity-Based Prompt Retrieval
One of the most innovative aspects of IMFL is how it uses human labels. Instead of just using them as training data for the final model, human-labeled pairs are stored in a retrieval system. When the LLM is tasked with labeling a new batch of data, the system retrieves the most semantically similar human-labeled examples to serve as in-context demonstrations. This "few-shot" prompting significantly improves the LLM's accuracy by providing it with domain-specific context and expert reasoning patterns.
3. Variable Batch Size (Budget Decay)
Timing is as important as selection. IMFL utilizes a geometric decay for the human annotation budget, front-loading human effort in the early stages of the process. By securing high-quality labels early, the system builds a robust retrieval pool for the LLM annotator, preventing the "cold start" problem where early errors in LLM labeling propagate through the training process.
Experimental Evaluation
The researchers evaluated the IMFL framework across four challenging benchmarks in the financial and medical sectors:
- Financial News: Financial Phrasebank (FPB) for sentiment analysis and Headline for gold market classification.
- Biomedical Research: PubMedQA (Yes/No/Maybe) and MedQA (USMLE-style questions).
Key Results
The experiments demonstrated that IMFL provides a massive efficiency boost over traditional labeling methods:
- Budget Efficiency: Using a total budget of 1,000 labels (200 human + 800 LLM), IMFL consistently outperformed a baseline using 600 human labels (3× the human budget).
- Performance Parity: On datasets like Headline and PubMedQA, IMFL achieved performance levels nearly identical to a model trained on 1,000 human labels (5× the human budget).
- LLM Accuracy Gains: The similarity-based prompt retrieval increased the accuracy of the LLM annotator (GPT-3.5) by 6–9% compared to zero-shot or random few-shot baselines.
- Domain Impact: In the complex MedQA task, the framework showed a +19.95% absolute gain over using LLM labels alone, highlighting the necessity of human-in-the-loop systems for high-stakes domains.
Discussion and Future Work
IMFL demonstrates that human labels have compound value: they serve as both direct training signals and as "teaching tools" for automated annotators. By prioritizing diversity and hardness through the EEQ strategy and stabilizing the process with front-loaded human effort, IMFL offers a viable path for organizations to build high-performing, domain-specific models without the need for massive human workforces.
Future research directions include refining the cost model to account for the varying difficulty of different samples and exploring the use of even more capable models (like GPT-4 or Claude 3) as the low-fidelity source. Additionally, investigating how the framework handles dynamic or evolving data pools could further enhance its utility in real-world, production environments.
Summary updated