BARRIER: Bounded Activation Regions for Robust Information Erasure

Jan Miksa, Patryk Krukowski, Przemysław Spurek, Dawid Damian Rymarczyk, Marcin SenderaarXiv 2026 · arXiv

TL;DR

Traditional machine unlearning methods often cause "collateral damage" by accidentally erasing benign knowledge along with the targeted data. BARRIER solves this by shifting the focus from static model weights to the dynamic geometry of hidden-layer activations, using Interval Arithmetic to draw a strict mathematical boundary around the concepts to be forgotten. This approach provides formal guarantees that protect the remaining data, allowing for highly aggressive and precise concept erasure without risking model collapse.

What Problem Does This Paper Address?

As large-scale generative AI models are deployed, the ability to precisely remove specific data—such as copyrighted material, biased content, or Not Safe For Work (NSFW) concepts—has become a regulatory and operational necessity. This process is known as machine unlearning.

However, existing unlearning techniques suffer from a critical bottleneck: they primarily perform "parameter surgery" on static network weights. Because different concepts are deeply entangled within a model's weight space, modifying weights to erase a target concept often inadvertently suppresses neutral, unrelated representations. To avoid this collateral damage, practitioners are forced to use conservative updates, which frequently result in incomplete erasure or model degradation.

Figure 1: Although current leading unlearning methods have little issue with concept or object removal in gener- ative models, they often introduce significant changes to benign samples. This contradicts the core philosophy of ma- chine unlearning, where we want to precisely erase targeted knowledge while strictly preserving the rest of the model’s capabilities. To address this, we introduce BARRIER, a novel framework that protects non-targeted representations with mathematical guarantees. We present NSFW concept unlearning in Flux.1 [dev] [33] model, demonstrating that BARRIER successfully removes explicit content without cor- rupting safe generations, as opposed to EraseAnything [16].

Methods

BARRIER (Bounded Activation Regions for Robust Information Erasure) introduces a paradigm shift by intervening in the dynamic geometry of hidden-layer activations rather than static weights.

The framework operates in three main steps:

  1. Low-Rank Subspace Extraction: High-dimensional hidden-layer activations from the "forget set" are centered and projected into a lower-dimensional manifold using truncated Singular Value Decomposition (SVD). This isolates the principal directions of variation for the target concept.
  2. Interval Arithmetic (IA) Bounding: Within this projected subspace, BARRIER defines a strict "forget interval" (a bounding hypercube) based on percentiles of the forget representations. The space outside this hypercube is designated as the invariant domain (the complement).
  3. Total Protection Loss: BARRIER introduces a specialized loss function ($L_{\text{Protect}}$) that mathematically bounds the model's functional drift on the interval complement. This ensures that any parameter updates driven by the unlearning objective occur exclusively within the forget region, leaving the rest of the activation space stable.
Figure 2: BARRIER can be integrated at arbitrary layers of a neural network to perform MU. Within projected space, a protection loss leverages Interval Arithmetic to constrain parameter updates: unlearning is driven exclusively within the targeted forget interval, while representations outside this designated region (the interval complement) remain strictly stabilized. As a result, BARRIER enables targeted concept removal while providing formal theoretical guarantees on bounded functional drift for protected representations. Note that, to ensure computational tractability in high dimensions, a truncated SVD is applied to the activation space to identify a low-rank subspace capturing the dominant directions of the forget set.

Datasets

The authors evaluated BARRIER across both classification and generative tasks using several standard benchmarks:

  • CIFAR-10 & CIFAR-100: Used for evaluating class-wise forgetting and random data forgetting in image classification (via ResNet-18) and class unlearning in generative models (via DDPM).
  • Inappropriate Image Prompts (I2P): A dataset containing 4,703 adversarial prompts used to benchmark the erasure of NSFW content.
  • MS-COCO: Used to evaluate general model utility and image quality preservation (via FID and CLIP scores) after unlearning in Stable Diffusion v1.4 and Flux.1 [dev].
  • Imagenette: A 10-class subset of ImageNet used for large-scale class erasure verification in Stable Diffusion.

Key Insights and Ideas

  • Functional Stability over Weight Rigidity: Instead of freezing weights or restricting parameter updates empirically, BARRIER bounds the output deviation of layers. This turns knowledge preservation from an empirical heuristic into a formal optimization target.
  • Provable Safety Enables Aggressive Erasure: Because the protection loss provides a probabilistic tail bound on functional drift for retained data, developers can apply highly aggressive unlearning updates within the forget zone without fearing catastrophic model collapse.
  • Extreme Parameter Efficiency: By targeting specific activation layers (like the QKV projection matrices in attention blocks), BARRIER is entirely architecture-agnostic and requires updating only a tiny fraction of the model's total parameters (e.g., just 0.05% in ResNet-18).
Figure 3: Preprocessing stage for BARRIER. In this part, we identify forget subspaces which serves for unlearning and its compliment that assures preservation of other model’s capabilities. Note that, the low-rank projection is done via SVD.

Key Results

BARRIER demonstrates state-of-the-art trade-offs between concept erasure and utility preservation:

Generative Class Unlearning (DDPM on CIFAR-10)

BARRIER achieved an Unlearning Accuracy (UA) of 95.20% and an outstanding Test Accuracy (TA) of 98.62%, while maintaining a low Fréchet Inception Distance (FID) of 12.02, outperforming weight-centric baselines.

NSFW Concept Erasure

  • Stable Diffusion v1.4: BARRIER successfully reduced NudeNet safety violations on the I2P dataset from 743 down to 18, while beautifully preserving the prompt's benign context and overall visual quality.
  • Flux.1 [dev]: On this massive state-of-the-art model, BARRIER reduced NudeNet detections from 605 to 171, significantly outperforming alternative frameworks like EraseAnything, which severely corrupted safe generations.
Figure 4: Qualitative comparison of NSFW con- cept unlearning across different methods in Sta- ble Diffusion v1.4 using adversarial prompts from the I2P dataset. BARRIER successfully mitigates explicit content, generating safe and visually coherent alternatives while gracefully preserving the original prompt’s benign context. For extended comparison see Fig 8.
Figure 5: Qualitative comparison of NSFW concept unlearning across different methods on Flux.1 [dev] using adversarial prompts from the I2P dataset. BARRIER successfully limits the generation of explicit content while maintaining overall visual fidelity.

Image Classification (CIFAR-10/100)

For class-wise forgetting on CIFAR-10 using ResNet-18, BARRIER achieved 100% Unlearning Accuracy while maintaining 97.72% Retain Accuracy, all while updating a mere 0.05% of the total parameters.

Limitations and Future Work

While BARRIER proves highly effective by protecting specific deep layers, the authors note that simply increasing the number of constrained layers does not automatically yield better results. The effectiveness of the interval protection heavily depends on where semantic representations are most densely concentrated within a specific network architecture.

As a next step, the authors suggest developing adaptive, automated strategies to dynamically identify the most informative layers for activation bounding, removing the need for manual layer selection.

Summary updated

Abstract

Machine unlearning has reached a critical bottleneck. As traditional weight-space interventions focus primarily on erasing targeted concepts, they often fail to prevent the unintended suppression of other significant representations. This leads to substantial collateral damage, with essential knowledge being forgotten, because these methods lack formal mathematical guarantees for the preservation of neutral concepts. To avoid degradation, they are frequently forced into conservative updates. We propose BARRIER (Bounded Activation Regions for Robust Information Erasure), a paradigm-shifting framework that shifts the locus of intervention from static model weights to the dynamic geometry of hidden-layer activations. Unlike existing methods, BARRIER employs Interval Arithmetic (IA) on SVD-based projections of the activation space to encapsulate the specific target region within a bounding hypercube. By driving unlearning updates exclusively within this forget interval and mathematically bounding the model response on the complement, we ensure rigorous protection of the retain distribution. This geometric construction transforms the preservation of knowledge from an empirical heuristic into a formal optimization target with a probabilistic tail bound on functional drift. Crucially, this stability permits highly aggressive unlearning updates within the forget region. Empirical evaluations demonstrate that BARRIER matches state-of-the-art trade-offs across classifiers and diffusion models, maximizing targeted concept erasure while safeguarding the integrity of all other representations. Our code is available at https://github.com/OneAndZero24/BARRIER.
BibTeX
@misc{miksa2026barrier,
  author = {Jan Miksa and Patryk Krukowski and Przemysław Spurek and Dawid Damian Rymarczyk and Marcin Sendera},
  title = {{BARRIER: Bounded Activation Regions for Robust Information Erasure}},
  year = {2026},
  eprint = {2605.15737},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.15737},
}

Loading related content...

Figure