Introduction
Image Forgery Localization (IFL) is the task of identifying manipulated regions within an image at the pixel level. As digital manipulation techniques, including generative AI, become increasingly sophisticated, the need for robust IFL methods has grown. Current approaches often struggle to balance the extraction of comprehensive tampering clues with the exclusion of task-irrelevant noise, such as JPEG compression artifacts or camera processing traces. This noise often leads to false positives or missed detections of subtle forgeries.
To address this challenge, the authors propose SUMI-IFL, a novel framework grounded in information theory. By explicitly applying "sufficiency" and "minimality" constraints to the feature learning process, SUMI-IFL maximizes the capture of task-relevant information while bottlenecking irrelevant noise.
Methodology
The SUMI-IFL framework optimizes feature representation through two primary constraints derived from Information Bottleneck (IB) theory.
Sufficiency-View Constraint
The first stage of the pipeline is the Feature Extraction Network, which aims to capture a diverse and comprehensive set of forgery clues. The model utilizes a multi-stream architecture with three parallel U-Net backbones, each specialized via custom attention mechanisms:
- Channel Attention Block (CAB): Focuses on color and texture inconsistencies.
- Spatial Attention Block (SAB): Targets structural anomalies.
- Pixel Attention Block (PAB): Examines local detail modifications.
The sufficiency-view constraint ensures that the fused latent features retain maximum mutual information with the ground-truth forgery mask, preventing the loss of critical tampering signals during extraction.
Minimality-View Constraint
Following extraction, the Feature Reasoning Network applies the minimality-view constraint. This stage acts as an information bottleneck, distilling the latent features into a concise representation ($Z$). It forces the network to discard superfluous, task-unrelated information—such as standard image noise—while preserving the essential forgery data. To enhance generalization and prevent the network from learning shortcuts, the authors employ a mask-guided encoder-decoder structure and inject point noise into the ground-truth mask during training.
Experimental Results
SUMI-IFL was evaluated on several standard benchmarks (DEFACTO-12, CASIAv2, SSRGFD) and challenging cross-dataset scenarios (NIST16, CIMD).
Performance and Robustness
- State-of-the-Art Accuracy: The model achieved leading F1 scores across multiple datasets, notably reaching 0.9249 on DEFACTO-12 and 0.7604 on CASIAv2. On the SSRGFD dataset, it outperformed the runner-up by 15.7%, demonstrating its superior ability to filter out complex reconstruction artifacts.
- Generalization: SUMI-IFL showed strong performance on unseen data, improving upon existing methods by 9.7% on the NIST16 benchmark.
- Ablation Studies: Removing the sufficiency and minimality constraints resulted in a 9.8% drop in F1 score on DEFACTO-12, confirming that these information-theoretic constraints are the primary drivers of the model's success.
Conclusion
SUMI-IFL introduces a mathematically grounded approach to image forensics by leveraging Information Bottleneck theory. By balancing the need for comprehensive feature extraction with strict noise filtering, the framework achieves significant improvements in localization accuracy and robustness. Future research may focus on reducing the computational complexity of the multi-stream architecture or extending these principles to video-based forgery detection.
Summary updated