MFL-Owner: Ownership Protection for Multi-modal Federated Learning via Orthogonal Transform Watermark

Keke Gai, Dongjue Wang, Jing Yu, Mohan Wang, Liehuang Zhu, Qi WuAAAI 2025 · DOI

Introduction

Multi-modal Federated Learning (MFL) has emerged as a cornerstone for collaborative AI, enabling multiple parties to train powerful models like CLIP without exposing sensitive local data. These models are frequently deployed as Embedding-as-a-Service (EaaS), where clients pay for vector representations. However, this deployment model introduces a significant security risk: model extraction attacks. An adversary can query the service to obtain embeddings and train a surrogate "student" model that replicates the original's functionality. Proving ownership and identifying the source of a leak in a multi-party federated environment remains a critical challenge.

The Challenge of Multi-modal Watermarking

Traditional watermarking often relies on "backdooring," where a model is trained to produce specific outputs for secret trigger inputs. In the context of MFL, these methods face two major hurdles. First, they often degrade the model's primary task performance by distorting the embedding space. Second, they struggle with traceability—the ability to distinguish which specific client among many might have leaked the model.

Proposed Method: MFL-Owner

To address these issues, researchers proposed MFL-Owner, a framework that utilizes an Orthogonal Transform Watermark. The key innovation is decoupling the watermark injection from the federated training process.

Decoupled Injection and Orthogonal Constraints

Instead of modifying the model during training, MFL-Owner adds a linear transformation layer ($W$) after the visual and language encoders once the global model is finalized. By enforcing an orthogonal constraint ($W^T W \approx I$), the transformation acts as an isometry. This preserves the geometric relationships (distances and angles) between embeddings, ensuring that the model's original logic and performance on downstream tasks remain virtually unchanged.

Overview of the MFL-Owner framework, illustrating the stages of Federated Learning Training, Watermark Injection, and Ownership Verification.

Unique Trigger Sets and Traceability

For traceability, the server generates a unique "trigger set" for each participating client. These sets consist of Out-of-Distribution (OoD) images blurred with Gaussian noise and unique text descriptions synthesized by a Large Language Model (LLM). During verification, if a suspected stolen model reacts specifically to Client A's trigger set, ownership is confirmed, and Client A is identified as the leaker.

Experimental Evaluation

The framework was tested on standard benchmarks including Flickr30k for retrieval and ImageNet-1k for classification.

Performance and Detection

MFL-Owner demonstrated remarkable efficiency. While baseline methods often caused accuracy drops of up to 35%, MFL-Owner maintained performance with negligible decreases (0.1% to 2.8%). Furthermore, it achieved a high Detection Rate (DR) of 97.11%, proving its reliability in identifying unauthorized model replicas.

Traceability Results

The effectiveness of the traceability mechanism is visualized through distance heatmaps. The clear diagonal pattern indicates that the model responds uniquely to the specific trigger set assigned to each client, with minimal cross-interference.

Heatmaps of cosine and Euclidean distances demonstrating the clear diagonal effect required for accurate client traceability.

Conclusion

MFL-Owner provides a robust and practical solution for protecting the intellectual property of multi-modal models in federated settings. By leveraging orthogonal transforms, it achieves a balance between security, traceability, and model utility that previous backdoor-based methods could not reach. Future work will explore making these watermarks more resilient to adaptive adversaries and continuous model updates.

Summary updated

BibTeX
@inproceedings{gai2025mfl,
  author = {Keke Gai and Dongjue Wang and Jing Yu and Mohan Wang and Liehuang Zhu and Qi Wu},
  title = {{MFL-Owner: Ownership Protection for Multi-modal Federated Learning via Orthogonal Transform Watermark}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i3.32313},
}
Figure