On the Opportunities and Challenges of Prefill-to-Autoregressive Cross-Model Transfer

Zhenyi Shen, Junru Lu, Wei Liu, Jiangnan Ye, Jiazheng Li, Di Yin, Xing Sun, Tameem Adel, Yali Du, Lin Gui, Yulan HearXiv 2026 · arXiv

Large language models (LLMs) present a frustrating trade-off: the massive models we need for complex reasoning are agonizingly slow and expensive to run during the generation (decoding) phase. Conversely, small models are lightning-fast but often "hallucinate" or fail on difficult tasks.

A new paper, "On the Opportunities and Challenges of Prefill-to-Autoregressive Cross-Model Transfer," explores a middle ground called PACT. In this setup, a large "sender" model processes the initial prompt (the prefill stage), and its internal "thoughts" (latent representations) are passed to a much smaller "receiver" model that handles the actual response generation.

TL;DR

The authors introduce PLatour, a high-performing, open-source framework for large-to-small model collaboration that uses reinforcement learning to bridge models. Crucially, they also propose PACT-Check, a rigorous evaluation suite that reveals a startling truth: many performance gains in these systems come from the "bridge" acting as a simple adapter rather than actual information transfer from the larger model.

What Problem Does This Paper Address?

The core challenge is balancing capability and inference cost. While PACT systems like the closed-source Mostik have shown promise, the field lacks open-source frameworks and, more importantly, a way to tell if the models are actually communicating.

If you connect a 70B model to a 7B model and performance goes up, is it because the 7B model "understood" the 70B model's hints? Or did the trainable bridge between them simply act as a clever fine-tuning mechanism (like LoRA) that improved the 7B model independently of the sender? This paper seeks to decouple genuine information transfer from receiver-side adaptation.

Methods

The authors introduce PLatour (Prefill-only LATent cOmmUnication for Reasoning). Its architecture is elegantly simple:

  • Linear Bridges: It connects layers of the sender and receiver at corresponding relative depths using zero-initialized linear projections.
  • Direct RL Training: Unlike previous methods that use complex distillation or alignment, PLatour uses Group Relative Policy Optimization (GRPO) to train the bridges directly on task rewards.
  • Hidden State Transfer: Instead of transferring Key-Value (KV) caches, it transfers hidden states, making it compatible with modern hybrid-attention architectures.
Overview of the PLatour architecture, showing how linear projectors map sender states into the receiver's residual stream.

To evaluate these systems, they propose PACT-Check, which includes four "sanity checks":

  1. Dummy Transfer: Replacing question-specific sender info with a fixed template.
  2. Irrelevant Transfer: Feeding the receiver info from a completely different question.
  3. Self-Communication: Having the receiver "talk" to itself.
  4. Trained Receiver: Comparing against a receiver fine-tuned with LoRA or full-parameter GRPO.

Datasets

The researchers tested PLatour across a wide array of benchmarks:

  • Reasoning & Science: ARC-C, MMLU-Redux, and Chemistry L3 (SciKnowEval).
  • Mathematics: GSM8K, MATH500, AIME (2022-2026), HMMT 2025, and IMO-AnswerBench.
  • Coding: LiveCodeBench v6, HumanEval, and MBPP.

Key Insights and Ideas

  • The "Spurious Bridge" Trap: On some tasks like GSM8K, the authors found that "Irrelevant Transfer" (giving the model info about a different question) performed almost as well as the real thing. This suggests the bridge was just acting as a task-specific adapter, not a communication channel.
  • Linear is Better: Surprisingly, simple linear projectors outperformed complex non-linear MLPs. The authors hypothesize that when models share a "Platonic" representation space, a simple linear map is enough to align them.
  • RL > Distillation: Training the bridge to maximize rewards (RL) was far more effective than trying to force the small model to mimic the big model's probability distributions (distillation), which often degrades performance in small models.
Figure 1: Opportunities and challenges of Prefill–to-Autoregressive Cross-model Transfer (PACT). (a) PLatour consistently outperforms the open-source C2C (Fu et al., 2026) across all generation tasks and surpasses the closed- source Mostik on math and coding, despite using a much smaller sender. (b) Superior benchmark performance does not necessarily imply effective information transfer from the sender. PACT-Check examines whether these gains depend on sender information through inference-time interventions and separately trained controls. On Chemistry, PLatour substantially outperforms all four sanity checks, providing evidence of meaningful information transfer. In contrast, on GSM8K, all four checks achieve comparable performance, suggesting that PLatour’s large gain there cannot be attributed to task-specific information from the sender.

Key Results

  • State-of-the-Art Performance: PLatour significantly outperformed existing open-source baselines like C2C.
  • Beating Closed-Source: Using a 35B sender, PLatour surpassed the reported results of the 753B-parameter Mostik on several math and coding benchmarks.
  • Efficiency: While PACT adds upfront latency during prefill, this cost is amortized over long generations. For responses over a few thousand tokens, the total inference time is nearly identical to running the small model alone.
Inference efficiency analysis showing that PACT total time approaches standalone receiver time as output length increases.

Limitations and Future Work

The authors noted training instability when scaling to extremely large senders (e.g., 397B parameters), where performance would occasionally collapse. They also emphasize that PACT only works well when the sender has a genuine knowledge advantage that the receiver is capable of exploiting—if the task is too easy for the receiver or too hard for the sender, the bridge defaults to being a simple adapter.

Future work includes exploring cross-family transfer (connecting models with entirely different architectures) and developing more "mechanistic" explanations of exactly what information is being sent across the bridge.

Summary updated

Abstract

Prefill-to-Autoregressive Cross-model Transfer (PACT), a form of latent communication between language models, combines a larger model's prompt representations with a smaller model's efficient generation, offering a promising way to balance capability and inference cost. However, existing systems provide limited support for independently evaluating large-small model collaboration, and improved task performance alone does not establish whether the receiver benefits from the sender's information. We introduce PLatour, an open-source PACT framework that connects frozen models through lightweight, zero-initialized linear bridges trained directly with task rewards and supports hybrid-attention architectures. PLatour significantly outperforms open-source baselines on generation tasks and surpasses Mostik's reported results on shared mathematics and coding benchmarks using a substantially smaller sender. However, controlled experiments show that the gains on some tasks can be reproduced by uninformative sender signals, self-communication, or receiver-only training. We find that trainable bridges can act as parameter-efficient adaptation mechanisms without exploiting useful sender information. We therefore propose PACT-Check to quantify how much of a trained PACT system's gain genuinely comes from communication. Finally, we discuss the opportunities and challenges of PACT systems. Code is available at https://github.com/zhenyi4/platour.
BibTeX
@misc{shen2026opportunities,
  author = {Zhenyi Shen and Junru Lu and Wei Liu and Jiangnan Ye and Jiazheng Li and Di Yin and Xing Sun and Tameem Adel and Yali Du and Lin Gui and Yulan He},
  title = {{On the Opportunities and Challenges of Prefill-to-Autoregressive Cross-Model Transfer}},
  year = {2026},
  eprint = {2611.06976},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.06976},
}
Figure