Abstract
The rise of AI-generated image editing tools has made localized forgeries increasingly realistic, posing challenges for visual content integrity. Although recent efforts have explored localized AIGC detection, existing datasets predominantly focus on object-level forgeries while overlooking broader scene edits in regions such as sky or ground. To address these limitations, we introduce BR-Gen, a large-scale dataset of 150,000 locally forged images with diverse scene-aware annotations, which are based on semantic calibration to ensure high-quality samples. BR-Gen is constructed through a fully automated Perception-Creation-Evaluation pipeline to ensure semantic coherence and visual realism. In addition, we further propose NFA-ViT, a Noise-guided Forgery Amplification Vision Transformer that enhances the detection of localized forgeries by amplifying forgery-related features across the entire image. NFA-ViT mines heterogeneous regions in images, i.e., potential edited areas, by noise fingerprints. Subsequently, an attention mechanism is introduced to compel the interaction between normal and abnormal features, thereby propagating the generalization traces throughout the entire image, allowing subtle forgeries to influence a broader context and improving overall detection robustness. Extensive experiments demonstrate that BR-Gen constructs entirely new scenarios that are not covered by existing methods. Take a step further, NFA-ViT outperforms existing methods on BR-Gen and generalizes well across current benchmarks.
Introduction
As generative models like Diffusion Models (SDXL, BrushNet) become more sophisticated, the visual "seams" of a fake image have become nearly invisible to the human eye. Most current research focuses on "full-image" generation or "object-level" edits (e.g., adding a person to a room). However, real-world forgeries often involve subtle modifications to "stuff"—amorphous regions like the sky, grass, or water.
Existing datasets suffer from a pervasive forgery region bias, focusing on salient objects and causing detectors to overfit to object-centric artifacts. This paper addresses these gaps by introducing the BR-Gen dataset and a novel detection architecture, NFA-ViT.
The BR-Gen Dataset Pipeline
To create a high-quality, diverse dataset of 150,000 images, the authors developed an automated Perception-Creation-Evaluation pipeline:
- Perception: Uses GroundingDINO and SAM2 to identify both "things" (objects) and "stuff" (sky, ground), while Qwen2.5-VL generates descriptive prompts for these regions.
- Creation: Employs five state-of-the-art inpainting methods (GAN-based: LaMa, MAT; Diffusion-based: SDXL, BrushNet, PowerPaint) to ensure the dataset is not tied to a single generative architecture.
- Evaluation: A multi-stage filter using metrics like BRISQUE (structural integrity) and CLIP scores (semantic alignment) ensures only realistic, high-quality fakes are included.
NFA-ViT: Noise-guided Forgery Amplification
The proposed NFA-ViT architecture uses a dual-branch approach to detect subtle inconsistencies that visual-only models might miss:
- Noise Branch: Extracts a "noise trace" using Noiseprint++, capturing low-level sensor and algorithmic artifacts that differ between real and AI-generated regions.
- Image Branch: A Vision Transformer (ViT) backbone for standard feature extraction.
- Noise-guided Amplification Attention (NAA): This core innovation uses the noise branch to identify potential edited areas and modulates the attention mechanism to "amplify" and "diffuse" these subtle forgery features across the entire image. This makes weak signals much more distinguishable to the classifier.
Key Results and Generalization
NFA-ViT demonstrates superior performance in both localization and robustness across multiple benchmarks. On the new BR-Gen dataset, it achieved an IoU (Intersection over Union) of 0.907, significantly outperforming previous state-of-the-art models like SparseViT (0.824).
Beyond the BR-Gen dataset, NFA-ViT shows impressive cross-domain robustness. When tested on the GRE dataset, it achieved a 0.774 IoU, outperforming Trufor (0.677). Despite its dual-branch complexity, the model remains efficient with an inference time of approximately 40ms per image and a memory footprint of 2.6 GB, making it viable for practical deployment.
Conclusion
By shifting focus toward "scene-level" forgeries and utilizing noise fingerprints to amplify subtle digital traces, this work provides a more robust framework for forensic analysis. The BR-Gen dataset and NFA-ViT architecture represent a significant step forward in detecting the next generation of localized AI-generated content.
Summary updated