Heirloom: Carrying a Phone's Personalization Across Base-Model Generations

Zhuomin Chen, Jingchao Ni, Qingkai Dong, Xu Zheng, Gabriel Hübner Lucchesi, Mo Sha, Wei Cheng, Dongsheng LuoarXiv 2026 · arXiv

TL;DR

Heirloom is an on-device system that migrates personalized LLM adapters across base-model upgrades without requiring the original user labels. By using the old personalized model as a teacher to distill knowledge into a new adapter, it preserves user preferences even when model architectures and vocabularies change, making seamless personalization persistent across phone generations.

What Problem Does This Paper Address?

Modern smartphones use on-device Large Language Models (LLMs) that learn your habits—like how you tag movies or style your messages—using small, efficient "adapters" (typically LoRA). However, these adapters are brittle; they are tied to a specific version of a base model. When a vendor pushes an OS update with a new base model (e.g., upgrading from Qwen3 to Qwen3.5), the old adapter often becomes incompatible due to changes in the model's layers, internal dimensions, or vocabulary.

Retraining a new adapter usually requires access to the user's labeled history. But for privacy reasons (data minimization), phones often discard these labels after the initial training. If the labels are gone and the old adapter doesn't "fit" the new model, the user loses all their personalization. Heirloom tackles this "migration gap" by transferring learned behavior between incompatible models directly on the device.

Figure 1. Heirloom migrates a personal adapter across a base-model upgrade. (a) The old personal model, the user’s adapter on the old base. (b) The vendor ships a new base that is incompatible with the old adapter, and retraining needs labels the phone did not keep. (c) Heirloom trains the new adapter to match the old model’s outputs on the stored inputs.

Methods

Heirloom uses a two-phase knowledge distillation approach to bridge the generational gap:

  1. Phase 1: Teacher Inference. The old personalized model (the "teacher") processes a set of prompts. These can be unlabeled past inputs stored on the phone, vendor-provided prompts, or even synthetic prompts generated by the new base model. The teacher records its predictions—either full probability distributions for classification tasks or generated text for open-ended tasks.
  2. Phase 2: Student Training. To save memory, the old model is unloaded. The new base model (the "student") is loaded with a fresh vendor-provided adapter. The student is then trained to match the teacher's recorded outputs.

Key Technical Innovations: * Cross-Vocabulary Distillation: Because different model generations often use different tokenizers, Heirloom distills probabilities over task-level labels (like "Action" or "Comedy") rather than raw tokens, providing a common interface between incompatible vocabularies. * Memory Management: By separating inference and training into two phases (see Figure 5 in the paper), Heirloom ensures that only one billion-parameter model is in the phone's RAM at any given time. * Label-Free Validation: Without ground-truth labels, the system uses "held-out agreement" with the teacher model to decide when to stop training, preventing overfitting to the teacher's quirks.

The structural changes between Qwen generations (e.g., layer counts and vocabulary size) that make direct adapter reuse impossible.

Datasets

The authors evaluated Heirloom using the LaMP (Language Model Personalization) benchmark, focusing on three distinct tasks:

  • Movie Tagging (LaMP-2M): Predicting which of 15 tags a specific user would give a film.
  • Product Rating (LaMP-3): Predicting a user's 1–5 star rating based on a product review.
  • Headline Generation (LaMP-4): Generating a news headline in a specific user's writing style.

The experiments covered upgrades across the Qwen and Phi model families, including transitions with significant architectural shifts.

Key Insights and Ideas

  • The Teacher's Confidence Matters: Matching the teacher's full probability distribution (how much it weighed "Dark Comedy" vs. "Horror") is significantly more effective than just matching its top choice. It captures the nuance of user preferences that single labels might miss.
  • Synthetic Prompts are a Viable Safety Net: If a phone deletes all past inputs for privacy, the new model can generate its own "hallucinated" task data for the old model to label. While not as good as using real history, this still recovers most of the personalization.
  • Weight Copying is Not Enough: Even in "refreshes" where the architecture stays the same but weights are updated, simply copying the old adapter leads to a significant drop in accuracy. Distillation remains the superior path for behavior preservation.
  • On-Device Feasibility: The system is designed to run on a phone's Neural Processing Unit (NPU), making it efficient enough to complete during a standard overnight charge.

Key Results

  • Accuracy Retention: On movie tagging, Heirloom maintained pre-upgrade accuracy across three major generation upgrades. Its performance was within 1.7 percentage points of the "gold standard" (retraining with the original private labels).
  • Stability: Heirloom achieved a lower negative flip rate than standard retraining. This means it was less likely to "break" things the old model already got right, as its explicit goal is to mimic the previous version's behavior.
  • Resource Cost: On a OnePlus 15, a full migration for a 4B-parameter model took 38 minutes and consumed 20% of the battery end-to-end.

Limitations and Future Work

  • History Caps: The system uses a cap on stored inputs (e.g., 128 or 512) to bound computation time. For users with massive histories, this cap can lead to a slight loss in accuracy compared to full-history retraining.
  • NPU Compatibility: The current training engine is optimized for specific architectures. Future work is needed to support more diverse model structures, such as those with fused projections or specialized position encodings.
  • Long-term Drift: The study looks at single-step upgrades. Investigating how personalization might "drift" or degrade over multiple successive generations of model migrations is a promising area for future research.

Summary updated

Abstract

On-device language models can learn a user's preferences through adapters trained on private data. However, a base-model upgrade may change the layers or vocabulary, leaving the personal adapter incompatible. Replacing it with a non-personal adapter loses much of the personalization, while retraining requires retaining the user's labeled history. We present Heirloom, a system that transfers a personal adapter to a new base model on the phone without retaining labels. Heirloom uses the old personal model as a teacher, collecting its predictions on stored unlabeled inputs, vendor-provided prompts, or prompts generated by the new base. It then unloads the old base and trains an adapter for the new model to match those predictions, keeping only one base model in memory at a time. We evaluate Heirloom across three generation upgrades from two model families, an architecture-preserving refresh, and three personalization tasks. On movie tagging, migration from stored inputs maintains pre-upgrade accuracy and differs from labeled retraining by at most 1.7 percentage points across the three upgrades. On a phone's neural processing unit (NPU), a fixed-length migration for the 4B upgrade takes 38 minutes and consumes 20% of the battery end to end, making the stored-input setting feasible during overnight charging.
BibTeX
@misc{chen2026heirloom,
  author = {Zhuomin Chen and Jingchao Ni and Qingkai Dong and Xu Zheng and Gabriel Hübner Lucchesi and Mo Sha and Wei Cheng and Dongsheng Luo},
  title = {{Heirloom: Carrying a Phone's Personalization Across Base-Model Generations}},
  year = {2026},
  eprint = {2611.00650},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.00650},
}
Figure