Abstract
Deep learning models often exhibit overconfidence when presented with out-of-distribution (OOD) data, posing significant risks for real-world deployment. Traditional solutions typically involve computationally expensive retraining or complex scoring functions that fail to create a distinct separation between in-distribution (ID) and OOD samples. This paper introduces Watermarking, a novel model reprogramming technique that adds a static, learned pattern to input images at test time. By modifying the data rather than the model parameters, Watermarking effectively "reprograms" fixed classifiers to enhance OOD detection. Experimental results across CIFAR and ImageNet benchmarks demonstrate that Watermarking significantly reduces the False Positive Rate (FPR95) and improves AUROC, even in challenging near-OOD scenarios.
Introduction
A fundamental challenge in deploying deep neural networks is their tendency to assign high-confidence predictions to categories they have never encountered. This lack of robustness to out-of-distribution data is a major hurdle in safety-critical applications such as autonomous driving and medical diagnostics. While existing methods focus on modifying model weights or designing post-hoc scoring functions, these approaches often suffer from high computational overhead or insufficient discriminative power.
This work explores a different paradigm: Model Reprogramming. Instead of altering the model, we can modify the input data to better align with the model's internal representations for the purpose of OOD detection. We propose a general methodology called Watermarking, which applies a static, optimized pattern to all inputs at inference time to amplify the distinction between known and unknown data.
Methodology
The Watermarking Framework
The core of the proposed method is the application of a watermark $w$—a static pattern with the same dimensions as the input image. At test time, every input $x$ is transformed into a watermarked version:
$$x' = x + w$$
Unlike instance-specific perturbations (such as those used in ODIN), which require a separate backpropagation pass for every test image, the watermark is a single, unified pattern applied globally, making it highly efficient for real-time deployment.
Optimization Strategy
The watermark is not random; it is optimized using a specific loss function designed to "anchor" the model's behavior. The optimization objective aims to:
- Maintain high confidence scores for watermarked ID data.
- Force the model to produce low confidence scores for the watermark pattern itself, using Gaussian noise as a proxy for OOD data.
To ensure the learned watermark is robust and generalizes across different data distributions, the authors employ Sharpness-Aware Minimization (SAM). SAM helps navigate the optimization landscape to find "flat" minima, preventing the watermark from becoming overly sensitive to small perturbations and ensuring stable performance at test time.
Scoring Functions
Watermarking is designed to be compatible with existing OOD scoring strategies. The study focuses on two primary functions:
- Softmax Scoring: Utilizing the maximum class probability.
- Free Energy Scoring: An energy-based approach that aligns more closely with the underlying data density, often providing better separation between ID and OOD samples.
Experimental Evaluation
Datasets and Benchmarks
The efficacy of Watermarking was evaluated using standard OOD benchmarks:
- In-Distribution (ID): CIFAR-10, CIFAR-100, and ImageNet-1k.
- Out-of-Distribution (OOD): iSUN, SVHN, Textures, Places365, and LSUN.
- Near-OOD: A challenging setup using CIFAR-10 as ID and CIFAR-100 as OOD to test the model's ability to distinguish between semantically similar but distinct classes.
Key Results
The addition of a watermark consistently widened the gap between ID and OOD scores across all tested configurations:
- FPR95 Reduction: On CIFAR-10, using Free Energy scoring, Watermarking reduced the False Positive Rate (at a 95% True Positive Rate) from 37.67% to 23.69%.
- Large-Scale Scalability: On ImageNet, the average FPR95 dropped from 54.93% to 40.50% using Softmax scoring, proving the method's viability for high-resolution, complex datasets.
- Near-OOD Performance: In the difficult CIFAR-10 vs. CIFAR-100 task, the AUROC improved from 55.47% to 65.69%, demonstrating that the watermark helps the model identify subtle differences between similar distributions.
Discussion and Insights
The "Edge" Effect
Visual analysis of the learned watermarks reveals an interesting structural property: the patterns tend to preserve the center of the image—where the primary object is typically located—while adding complex, high-frequency information to the edges. This suggests the watermark acts as an auxiliary signal that helps the model "anchor" its ID predictions while making OOD inputs appear more distinct from the learned ID manifold.
Orthogonality and Efficiency
Because Watermarking is a data-level manipulation, it is inherently orthogonal to model-level or score-level OOD methods (like ReAct or ODIN). It can be stacked with these techniques to provide an additive performance boost. Furthermore, its static nature ensures that the computational cost at inference time is negligible compared to methods requiring per-instance optimization.
Limitations
A notable trade-off is a slight decrease in standard classification accuracy. For example, on CIFAR-10, accuracy dropped from 94.84% to 93.49%. However, the authors note that using Free Energy scoring can mitigate this impact more effectively than Softmax scoring. Additionally, performance is sensitive to hyperparameters such as the noise standard deviation ($\sigma_1$) and the SAM constraint ($\rho$), requiring careful validation during the optimization phase.
Conclusion
Watermarking represents a shift in OOD detection research, moving from model modification to data reprogramming. By applying a static, learned pattern to inputs, we can significantly improve a model's ability to reject unknown data without the need for costly retraining. Future work will focus on refining the optimization objectives to eliminate the accuracy trade-off and exploring the application of watermarking to other modalities beyond computer vision.
Summary updated