Abstract
Protecting the intellectual property of Automatic Speech Recognition (ASR) models is challenging due to the unique temporal nature of sequence-to-sequence architectures. This paper presents a novel black-box watermarking framework that utilizes continuous speech patterns as triggers and linguistic steganography for natural-sounding target labels. The approach effectively bypasses the Connectionist Temporal Classification (CTC) alignment problem and resists label-detection evasion attacks. Experimental results on the DeepSpeech model demonstrate high fidelity, perfect watermark extraction, and significant robustness against model pruning and fine-tuning.
Introduction
The high cost of training state-of-the-art Deep Neural Networks (DNNs) makes intellectual property (IP) protection a critical concern. Black-box watermarking allows owners to verify ownership by querying a model with specific "trigger" inputs and observing the outputs. However, traditional watermarking techniques designed for classification tasks fail when applied to ASR systems.
Two primary obstacles exist: the CTC alignment problem, where localized triggers disrupt the model's conditional independence assumptions, and label-detection evasion attacks, where obvious copyright strings in the output (e.g., "Copyright 2022") can be easily filtered by attackers. This paper introduces a framework designed to overcome these hurdles while maintaining model performance.
Proposed Framework
The authors propose a three-stage framework to address these challenges: trigger generation, watermark embedding, and extraction.
1. Continuous Trigger Injection
To solve the CTC alignment issue, the framework overlays a specific "speech pattern" (e.g., a repeated audio clip of the owner's voice) across the entire duration of the input audio. This forces the model to map the entire sequence to the target label consistently, bypassing the conflicts that arise from localized noise.
2. Linguistic Steganography
To prevent evasion, target labels are generated using a DNN-based linguistic steganography tool (VAE-stega). This hides the secret copyright bit-string within natural-sounding sentences. Because these "stego texts" appear benign, an attacker cannot filter them out without risking the interception of legitimate user queries.
3. Embedding and Extraction
The ASR model is fine-tuned on a combination of clean audio and trigger audio/stego text pairs. Ownership is verified by querying the model with trigger audio and decoding the hidden message from the output text. If the Word Error Rate (WER) and Character Error Rate (CER) are below a predefined threshold, the watermark is extracted.
Experimental Results
The framework was evaluated using the DeepSpeech (v0.5.1) ASR system on the LibriSpeech dataset.
Fidelity and Integrity
The watermarking process maintains high model performance. The Word Error Rate (WER) on clean audio increased only slightly from 3.376% to 3.723% after embedding. Furthermore, the system demonstrates high integrity; unwatermarked models yield a 100% WER when queried with triggers, preventing false positive claims.
Robustness
A key strength of this framework is its resilience to common model modifications intended to remove watermarks, such as parameter pruning and fine-tuning.
As shown in Figure 3, the watermark remains perfectly extractable (0% BER) even when the model is subjected to extreme pruning (up to 90% of weights zeroed out) or extensive fine-tuning (up to 50 epochs). This indicates that the watermark is deeply embedded within the model's learned representations and is highly resistant to removal attacks.
Conclusion
This work introduces the first robust black-box watermarking solution tailored for ASR systems. By leveraging continuous speech patterns and steganographic labels, the framework provides a secure and high-fidelity method for IP protection. Future research may focus on further reducing the performance overhead and extending the method to non-CTC architectures like Transformers or Conformers.
Summary updated