Adaptive Equilibrium: Dynamic Weighting Framework for Generalized Interruption of DeepFake Models

Hongrui Zheng, Liejun Wang, Zhiqing GuoarXiv 2026 · OpenReview

TL;DR

This paper introduces the Adaptive Equilibrium Framework (AEF) to tackle the challenge of creating universal perturbations that effectively disrupt a wide range of deepfake generation models. AEF dynamically adjusts the influence of different models during the perturbation generation process, ensuring a balanced and robust defense against deepfakes.

What Problem Does This Paper Address?

Deepfake technology has advanced rapidly, posing significant threats like misinformation and identity theft. While active defense strategies aim to disrupt deepfake generation, creating a single "universal perturbation" that works effectively against diverse deepfake models remains a major hurdle. Existing methods often rely on static gradient normalization, which struggles with the architectural differences between models. This leads to an "interruption imbalance," where perturbations are effective against some models but fail against others, particularly more resistant ones. The paper aims to solve this imbalance by developing a method that can achieve a uniformly effective disruption across various deepfake architectures.

Methods

The core of the proposed AEF lies in its Adaptive Equilibrium Framework. This framework operates through two main branches:

  • Deep Feature Enhancement (DFE): This branch aims to corrupt semantic information by enhancing the variance between clean and adversarial features. It decouples the feature difference into three components:
  • Local Pattern Discrepancy ($d_{local}$): Targets instance-level statistics crucial for style-guided generators.
  • Global Statistical Discrepancy ($d_{global}$): Addresses attribute-conditioned networks by capturing distribution shifts in global mean and standard deviation.
  • Structural Semantic Discrepancy ($d_{structure}$): Disrupts channel-wise dependencies in attention-guided architectures. The final feature loss ($L_{feat}$) is a weighted sum of these discrepancies.

  • Adaptive Weight Tuning: This component dynamically calculates each model's contribution to the overall perturbation. It uses an Exponential Moving Average (EMA) to smooth historical loss values for each model, effectively tracking their resistance. A temperature-controlled Softmax function then assigns dynamic weights ($w_i$) to each model's loss, giving higher weights to more resistant models. This weighted sum of losses ($L_{global}$) is then used for unified backpropagation to update the perturbation. This approach shifts the optimization focus from an average-case problem to finding a dynamic balance, driving the perturbation towards a uniformly effective equilibrium.

Datasets

The paper utilizes the CelebA dataset for training the universal perturbation. For evaluation, it uses the CelebA test split, and additionally tests cross-dataset generalization on the LFW (Labeled Faces in the Wild) and FaceForensics++ (FF++O) datasets.

Key Insights and Ideas

  • Interruption Imbalance is Key: The paper correctly identifies the "interruption imbalance" as the fundamental bottleneck in universal deepfake disruption, stemming from architectural diversity.
  • Static Gradients are Insufficient: The authors demonstrate why static gradient averaging fails to generalize, as it gets dominated by susceptible models and ignores architectural conflicts.
  • Adaptive Weighting for Robustness: The core insight is that dynamically weighting models based on their resistance (tracked via EMA) is crucial for achieving uniform effectiveness. This allows the perturbation to target the "hardest" models.
  • Feature-Level Disruption is Multi-faceted: The DFE branch's approach of decoupling feature differences into local, global, and structural components provides a more comprehensive attack strategy than single-metric approaches.
  • Temperature Control for Balance: The temperature hyperparameter in the Softmax function offers a tunable knob to control the distribution of weights, allowing for a trade-off between focusing on the most resistant model and a broader disruption.

Key Results

  • Superior Interruption Performance: AEF consistently achieves higher SRmask (interruption success rate) and L2mask scores across various deepfake models and datasets compared to baselines like CMUA, FOUND, DWT, and TSDF. For example, AEF achieves an average SRmask of 99.88% on CelebA.
  • Robustness Across Architectures: The framework demonstrates strong performance even on challenging architectures like AttGAN and HiSD, maintaining high SRmask scores (e.g., 99.91% on AttGAN for CelebA).
  • High Imperceptibility: AEF maintains high SSIM and PSNR scores, indicating that the generated perturbations are visually imperceptible and do not degrade image quality significantly.
  • Computational Efficiency: AEF is significantly faster than some baselines, with a training time of approximately 0.23 hours on a single GPU, compared to over 5 hours for CMUA.
  • Effective Black-Box Transferability: AEF shows superior generalization equilibrium in black-box settings, outperforming baselines in disrupting unseen models.

Limitations and Future Work

  • Hyperparameter Tuning: While the paper explores hyperparameter sensitivity (e.g., feature shift $\alpha$ and temperature $T$), finding the optimal configuration might still require careful tuning for specific scenarios. The paper identifies $\alpha=0.8$ and $T=0.1$ as optimal.
  • Computational Cost of Ensemble: While AEF is more efficient than some baselines, the need to evaluate multiple models during training still incurs computational overhead.
  • Future Work: The paper implicitly suggests that further exploration of adaptive mechanisms and feature enhancement strategies could lead to even more robust and efficient deepfake disruption methods. The focus on achieving "adversarial equilibrium" is a promising direction for future research in robust defense against generative models.

Summary updated

BibTeX
@misc{zheng2026adaptive,
  author = {Hongrui Zheng and Liejun Wang and Zhiqing Guo},
  title = {{Adaptive Equilibrium: Dynamic Weighting Framework for Generalized Interruption of DeepFake Models}},
  year = {2026},
  url = {https://openreview.net/forum?id=vfSKrEtfcG},
}
Figure