Detecting Localized and Partial Image Forgeries

44 concepts37 papers2 proposals

Early deepfake and synthetic image detectors operated on a simple premise: an image is either entirely real or entirely fake. By training a binary classifier on whole images, models could easily pick up on global statistical anomalies or structural artifacts left behind by generative networks.

However, the landscape of digital manipulation has shifted from full-image generation to surgical, localized editing. Techniques like generative inpainting, face-swapping, and instruction-guided object modification allow bad actors to alter the semantic meaning of an image while leaving the vast majority of its pixels completely untouched. When standard global classifiers encounter these hybrid images, their performance drops precipitously—often to near-random accuracy.

Understanding why this happens, and how modern forensic frameworks overcome it, requires looking at a phenomenon known as signal drowning.

The Signal Drowning Problem

Why do models that achieve near-perfect accuracy on fully synthetic images fail when only a small region is modified? The core issue is that the overwhelming volume of authentic pixels masks the subtle, high-frequency traces of the forgery.

When a global classifier processes an image, it aggregates features across the entire spatial domain. As shown by Jiwen Lu et al. in NeXT-IMDL: Build Benchmark for NeXT-Generation Image Manipulation Detection & Localization, the massive proportion of pristine pixels effectively dilutes the statistical evidence of tampering. This is not just a passive dilution; it is an active suppression. Anirudh Sundara Rajan and Yong Jae Lee demonstrated in Stay-Positive: A Case for Ignoring Real Image Features in Fake Image Detection that standard detection architectures allow "realness" features from the authentic regions to counteract or suppress the localized "fakeness" artifacts during the model's decision logic.

Furthermore, analyzing an image as a single holistic unit means that discriminative signals in complex, localized regions are easily overwhelmed by simpler, high-area regions like backgrounds, a challenge highlighted by Hengrui Lou et al. in STD-FD: Spatio-Temporal Distribution Fitting Deviation for AIGC Forgery Identification. As a result, passive detection systems struggle to isolate subtle changes when the global image structure remains largely intact (Proactive Image Manipulation Detection by Vishal Asnani et al.).

Shifting from Global to Granular Analysis

To prevent authentic pixels from drowning out forgery signatures, researchers have shifted from holistic classification to region-specific, granular localization. The intuition is straightforward: if you isolate suspicious regions before aggregating features, the authentic background can no longer dilute the signal.

One prominent approach is to decompose the image into semantic fragments. For example, in facial forensics, Hao Dang et al. introduced attention mechanisms in On the Detection of Digital Face Manipulation to isolate manipulated facial regions from authentic ones, preventing pristine features from introducing noise. Similarly, Zehao Chen et al. designed a dual-layer attention framework in Attentive Semantic Exploring for Manipulated Face Detection that breaks faces down into distinct semantic components (like pupils or lips) to catch fine-grained inconsistencies that global models overlook.

Beyond faces, detecting partial modifications in general imagery requires fusing multiple forensic perspectives. Siyuan Cheng and Lingjuan Lyu developed a framework in CO-SPY: Combining Semantic and Pixel Features to Detect Synthetic Images by AI that adaptively combines high-level semantic analysis with low-level pixel artifacts, allowing the system to catch localized edits like selective object modification. By forcing the network to focus on region-specific anomalies, these methods ensure that localized generative traces remain detectable.

New Benchmarks for Semi-True Media

The development of localized forgery detection has been heavily accelerated by a transition in how datasets are constructed. Traditional benchmarks primarily evaluated full-image or uniform synthesis styles, creating a "benchmark illusion" where models appeared robust but failed on real-world, surgical edits.

To address this, recent datasets explicitly model the logical and regional nature of modern image editing. Zhihao Sun and Juan Cao introduced the Generative Regional Editing (GRE) dataset in Rethinking Image Editing Detection in the Era of Generative AI Revolution, which utilizes a pipeline where language models decide contextually plausible regional modifications executed by diffusion models. Similarly, the SEMI-TRUTHS dataset introduced by Anisha Pal et al. in Semi-Truths: A Large-Scale Dataset of AI-Augmented Images for Evaluating Robustness of AI-Generated Image detectors provides over a million AI-augmented images with fine-grained metadata to expose the brittleness of detectors optimized solely for fully generated content.

By evaluating models on benchmarks like UniAIDet: A Unified and Universal Benchmark for AI-Generated Image Content Detection and Localization (developed by the authors) and DF40 (DF40: Toward Next-Generation Deepfake Detection by Zhiyuan Yan et al.), which feature instruction-guided editing and feature-specific manipulations, the community has established a clear consensus: training models to localize fine-grained, regional inconsistencies is essential for generalizable and robust deepfake defense.

Key Foundational Work

Domain Adaptation in Multi-Channel Autoencoder based Features for Robust Face Anti-Spoofing (2019) — contributed 1 concept elements to this direction.
On the Detection of Digital Face Manipulation (2020) — contributed 1 concept elements to this direction.
Which Is Plagiarism: Fashion Image Retrieval Based on Regional Representation for Design Protection (2020) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_localized_forgery_detection_a9f08fef,
  title = {localized forgery detection},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/localized-forgery-detection/rdr_a9f08fef1e9b41c0bfbc3fd76ccab55e},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_localized_forgery_detection_a9f08fef_v1,
  title = {localized forgery detection},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/localized-forgery-detection/rdr_a9f08fef1e9b41c0bfbc3fd76ccab55e/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake