The False Promise of Imitating Proprietary LLMs

Arnav Gudibande, Eric Wallace, Charlie Snell, Xinyang Geng, Hao Liu, Pieter Abbeel, Sergey Levine, Dawn SongarXiv 2023 · DOI · arXiv

Introduction

The rapid rise of large language models (LLMs) like ChatGPT has sparked a wave of "Alpaca-style" models. These are typically open-source base models, such as LLaMA, fine-tuned on a small fraction of data generated by proprietary giants. Early anecdotal evidence suggested these "imitation models" could achieve near-parity with their teachers at a fraction of the cost. However, new research suggests this performance is largely a "mirage." While these models excel at mimicking the authoritative tone and stylistic quirks of their teachers, they fail to bridge the fundamental gap in factual knowledge, reasoning, and coding capabilities.

Methodology: Probing the Imitation Mirage

To investigate whether imitation is a legitimate shortcut to state-of-the-art performance, researchers systematically built a series of imitation models. They fine-tuned base models of varying scales—GPT-2 (1.5B), LLaMA (7B), and LLaMA (13B)—using ChatGPT outputs. Their approach spanned two main strategies:

  1. Broad-coverage imitation: Utilizing diverse datasets like ShareGPT-Mix (~50k dialogues), HC3 (~27k responses), and Discord bot logs (~10k examples) to capture ChatGPT’s general persona.
  2. Task-specific imitation: Training on synthetic data (NQ-synthetic) tailored to the Natural Questions benchmark to test if targeted imitation could improve factuality.

Evaluation was multi-faceted, moving beyond standard benchmarks to include blind pairwise human comparisons (Crowdworker Evaluation) and automated ranking by GPT-4.

The Style vs. Substance Gap

The core finding of the study is that imitation models are adept at mimicking style but not substance. They learn to adopt professional formatting, use bulleted lists, and maintain a confident, helpful persona. This "persona filter" effectively masks an underlying lack of knowledge.

Comparison of ChatGPT and Imitation Model responses, highlighting how the imitation model mimics the style but introduces factual errors.

As shown in the qualitative comparison above, the imitation model successfully replicates the structure of a ChatGPT response. However, a closer look reveals that while it "acts" like a smart model, it often fails to provide the same level of technical accuracy or nuanced understanding.

The Human Evaluation Trap

One of the most striking results was the discrepancy between human perception and objective performance. In blind tests, crowd workers initially rated the 13B imitation model's outputs as equal to or better than ChatGPT's approximately 70% of the time. This suggests that humans are easily swayed by fluency and confidence—traits that imitation models are very good at faking.

Quantitative results showing high human preference for imitation models despite declining accuracy on factual benchmarks like Natural Questions.

However, as the data illustrates, this high preference does not translate to actual capability. While human ratings remained high or even improved with more imitation data, performance on hard NLP benchmarks like MMLU (knowledge) and HumanEval (coding) showed zero to negative improvement. On the Natural Questions benchmark, broad-coverage imitation actually caused a regression in performance, as models learned to hallucinate with greater confidence.

The Base Model Bottleneck

The research highlights a critical scaling law: the "highest leverage action" for improving open-source AI is developing better base models through extensive pre-training, rather than collecting more imitation data. Increasing the amount of imitation data (from 0.3M to 150M tokens) yielded diminishing returns. In contrast, increasing the base model size from 7B to 13B parameters led to much more significant and genuine performance gains.

Conclusion

Fine-tuning on imitation data acts more like a stylistic overlay than a knowledge transfer mechanism. It teaches a model how to present information it already knows but does not effectively "extract" new capabilities from the teacher. For the open-source community, the path forward lies in the hard work of better pre-training and diverse, high-quality data collection, rather than the "false promise" of simple imitation. Future research may explore if techniques like Reinforcement Learning from Human Feedback (RLHF) can bridge this gap, but the fundamental capability bottleneck remains the base model's pre-training.

Summary updated

Abstract

An emerging method to cheaply improve a weaker language model is to finetune it on outputs from a stronger model, such as a proprietary system like ChatGPT (e.g., Alpaca, Self-Instruct, and others). This approach looks to cheaply imitate the proprietary model's capabilities using a weaker open-source model. In this work, we critically analyze this approach. We first finetune a series of LMs that imitate ChatGPT using varying base model sizes (1.5B--13B), data sources, and imitation data amounts (0.3M--150M tokens). We then evaluate the models using crowd raters and canonical NLP benchmarks. Initially, we were surprised by the output quality of our imitation models -- they appear far better at following instructions, and crowd workers rate their outputs as competitive with ChatGPT. However, when conducting more targeted automatic evaluations, we find that imitation models close little to none of the gap from the base LM to ChatGPT on tasks that are not heavily supported in the imitation data. We show that these performance discrepancies may slip past human raters because imitation models are adept at mimicking ChatGPT's style but not its factuality. Overall, we conclude that model imitation is a false promise: there exists a substantial capabilities gap between open and closed LMs that, with current methods, can only be bridged using an unwieldy amount of imitation data or by using more capable base LMs. In turn, we argue that the highest leverage action for improving open-source models is to tackle the difficult challenge of developing better base LMs, rather than taking the shortcut of imitating proprietary systems.
BibTeX
@misc{gudibande2023false,
  author = {Arnav Gudibande and Eric Wallace and Charlie Snell and Xinyang Geng and Hao Liu and Pieter Abbeel and Sergey Levine and Dawn Song},
  title = {{The False Promise of Imitating Proprietary LLMs}},
  year = {2023},
  eprint = {2305.15717},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2305.15717},
}
Figure