AWM: Accurate Weight-Matrix Fingerprint for Large Language Models

Boyi Zeng, Lin Chen, Ziwei He, Xinbing Wang, Zhouhan Lin

TL;DR

AWM (Accurate Weight-Matrix Fingerprint) is a training-free, ultra-fast white-box fingerprinting method that reliably verifies Large Language Model (LLM) lineage in under 30 seconds. By leveraging the Linear Assignment Problem and unbiased Centered Kernel Alignment, it achieves perfect accuracy in distinguishing derived models from independent ones, demonstrating absolute robustness against aggressive post-training modifications like pruning, upcycling, and extensive continued pretraining.

What Problem Does This Paper Address?

Training a state-of-the-art LLM from scratch requires millions of dollars in compute and massive datasets, making a model's weights highly valuable intellectual property (IP). To circumvent licensing terms or claim false originality, bad actors might take an existing open-source base model, apply heavy post-training modifications, and re-release it as a completely new, proprietary model.

Detecting this "model laundering" is incredibly difficult. Standard IP protection methods like watermarking require extra training that can degrade model performance and often fail to survive aggressive fine-tuning. Existing fingerprinting methods fall short too: they either suffer from high false-positive rates (falsely accusing innocent developers) or completely lose track of a model's lineage if it undergoes extensive continued pretraining (e.g., being trained on trillions of additional tokens).

The authors address this gap by creating a high-fidelity, computationally efficient verification method that requires no additional training and leaves model performance completely untouched.

Methods

The authors introduce AWM, a white-box fingerprinting technique grounded in the mathematical constraints of transformer architectures. They prove that key components like RMSNorm and Rotary Position Embeddings (RoPE) severely restrict how an attacker can manipulate weights without destroying the model's performance.

AWM operates via a elegant two-stage pipeline:

  1. Permutation and Signature Extraction: The method aligns the word embeddings of two models over their shared vocabulary. By framing this alignment as a Linear Assignment Problem (LAP) and solving it using the Hungarian algorithm, AWM effectively neutralizes any malicious or accidental token permutations and sign flips.
  2. Weight Similarity Assessment: It compares the Query ($Q$) and Key ($K$) attention matrices layer-by-layer using an unbiased variant of Centered Kernel Alignment (UCKA). Because CKA is inherently invariant to orthogonal transformations and constant rescaling, it naturally bypasses complex parameter manipulations (like RoPE-induced rotations or structural pruning) without needing to explicitly reconstruct the transformation matrices.

The entire process is incredibly lightweight, executing in less than 30 seconds on a single commodity NVIDIA 3090 GPU.

Datasets

To rigorously evaluate AWM, the authors curated a comprehensive testbed of 150 model pairs:

  • 60 positive pairs (base model and its derived offspring), evenly split across six demanding post-training modification categories (10 pairs each): Supervised Fine-Tuning (SFT), Continual Pre-training (up to 5.5 Trillion tokens), Upcycling (converting a dense model into a Mixture-of-Experts), Multi-Modal extensions, Reinforcement Learning (RL), and Architectural Pruning.
  • 90 negative pairs consisting of completely independent, unrelated models to evaluate the risk of false positives.

The evaluation spans a wide variety of popular open-source model families, including LLaMA2 (7B, 13B, 70B), Llama3-8B, Mistral-7B, Gemma, and Qwen.

Key Insights and Ideas

  • Architecture as a Cage: The core insight of the paper is that an LLM's structural components act as a constraint system. If a derived model wants to maintain the high performance of its base model, any weight manipulations must conform to strict algebraic rules dictated by components like RMSNorm and residual connections.
  • The Vulnerability of RMSNorm: The authors mathematically demonstrate that RMSNorm commutes with constant scaling, permutations, and sign flips. This predictability allows them to cleanly reverse embedding alterations using standard combinatorial optimization.
  • Parameter-Free Invariance: Instead of trying to guess or explicitly reconstruct how an attacker might have rotated or pruned weight matrices, utilizing unbiased CKA provides a parameter-free shortcut. Its mathematical invariance handles these complex structural shifts automatically.

Key Results

  • Perfect Classification: AWM achieved a flawless score of 1.000 across all core evaluation metrics, including Area Under the Curve (AUC), partial AUC (pAUC for False Positive Rate < 0.05), and True Positive Rate at a 1% False Positive Rate.
ROC curves showing perfect AUC for AWM compared to baselines.
  • Zero False Accusations: While representation-based baselines like REEF frequently yield dangerously high similarity scores (>80%) for completely unrelated models, AWM maintained near-zero similarity scores for independent pairs (averaging just 0.49% for 7B models), eliminating the risk of false IP theft accusations.
Table showing high similarity for offspring models and negligible similarity for independent models.
  • Extreme Robustness: Even in extreme cases where prior state-of-the-art methods like HuRef failed—such as after a model underwent 5.5T tokens of continued pretraining—AWM maintained massive statistical separation margins (Z-scores > 135), proving its resilience against aggressive post-training.

Limitations and Future Work

  • White-Box Constraint: AWM is strictly a white-box fingerprinting method. It requires direct access to the model's internal weight matrices (specifically embeddings, Q, and K matrices). Consequently, it cannot be used to audit suspect models that are only accessible via closed APIs (black-box scenarios).
  • Future Work: The authors note that extending these structural invariance insights to black-box settings or exploring defenses against even more chaotic, non-functional weight obfuscation techniques remain open and promising directions for future research.

Summary updated

Abstract

Protecting the intellectual property of large language models (LLMs) is crucial, given the substantial resources required for their training. Consequently, there is an urgent need for both model owners and third parties to determine whether a suspect LLM is trained from scratch or derived from an existing base model. However, the intensive post-training processes that models typically undergo-such as supervised fine-tuning, extensive continued pretraining, reinforcement learning, multi-modal extension, pruning, and upcycling-pose significant challenges to reliable identification. In this work, we propose a training-free fingerprinting method based on weight matrices. We leverage the Linear Assignment Problem (LAP) and an unbiased Centered Kernel Alignment (CKA) similarity to neutralize the effects of parameter manipulations, yielding a highly robust and high-fidelity similarity metric. On a comprehensive testbed of 60 positive and 90 negative model pairs, our method demonstrates exceptional robustness against all six aforementioned post-training categories while exhibiting a near-zero risk of false positives. By achieving perfect scores on all classification metrics, our approach establishes a strong basis for reliable model lineage verification. Moreover, the entire computation completes within 30s on an NVIDIA 3090 GPU. The code is available at https://github.com/LUMIA-Group/AWM.
BibTeX
@inproceedings{zeng2026awm,
  author = {Boyi Zeng and Lin Chen and Ziwei He and Xinbing Wang and Zhouhan Lin},
  title = {{AWM: Accurate Weight-Matrix Fingerprint for Large Language Models}},
  booktitle = {International Conference on Learning Representations},
  year = {2026},
  url = {https://openreview.net/forum?id=fDC5WeLeqh},
}
Figure