SUMI-IFL: An Information-Theoretic Framework for Image Forgery Localization with Sufficiency and Minimality Constraints

Ziqi Sheng, Wei Lu, Xiangyang Luo, Jiantao Zhou, Xiaochun Cao

Introduction

Image Forgery Localization (IFL) is the task of identifying manipulated regions within an image at the pixel level. As digital manipulation techniques, including generative AI, become increasingly sophisticated, the need for robust IFL methods has grown. Current approaches often struggle to balance the extraction of comprehensive tampering clues with the exclusion of task-irrelevant noise, such as JPEG compression artifacts or camera processing traces. This noise often leads to false positives or missed detections of subtle forgeries.

To address this challenge, the authors propose SUMI-IFL, a novel framework grounded in information theory. By explicitly applying "sufficiency" and "minimality" constraints to the feature learning process, SUMI-IFL maximizes the capture of task-relevant information while bottlenecking irrelevant noise.

Methodology

The SUMI-IFL framework optimizes feature representation through two primary constraints derived from Information Bottleneck (IB) theory.

Overview of the SUMI-IFL pipeline, detailing the overall architecture (a) and the conceptual diagrams for the Sufficiency-view (b) and Minimality-view (c) constraints.

Sufficiency-View Constraint

The first stage of the pipeline is the Feature Extraction Network, which aims to capture a diverse and comprehensive set of forgery clues. The model utilizes a multi-stream architecture with three parallel U-Net backbones, each specialized via custom attention mechanisms:

  • Channel Attention Block (CAB): Focuses on color and texture inconsistencies.
  • Spatial Attention Block (SAB): Targets structural anomalies.
  • Pixel Attention Block (PAB): Examines local detail modifications.

The sufficiency-view constraint ensures that the fused latent features retain maximum mutual information with the ground-truth forgery mask, preventing the loss of critical tampering signals during extraction.

Detailed view of the multi-stream feature extraction network, showing the parallel processing of channel, spatial, and pixel-level features.

Minimality-View Constraint

Following extraction, the Feature Reasoning Network applies the minimality-view constraint. This stage acts as an information bottleneck, distilling the latent features into a concise representation ($Z$). It forces the network to discard superfluous, task-unrelated information—such as standard image noise—while preserving the essential forgery data. To enhance generalization and prevent the network from learning shortcuts, the authors employ a mask-guided encoder-decoder structure and inject point noise into the ground-truth mask during training.

Experimental Results

SUMI-IFL was evaluated on several standard benchmarks (DEFACTO-12, CASIAv2, SSRGFD) and challenging cross-dataset scenarios (NIST16, CIMD).

Performance and Robustness

  • State-of-the-Art Accuracy: The model achieved leading F1 scores across multiple datasets, notably reaching 0.9249 on DEFACTO-12 and 0.7604 on CASIAv2. On the SSRGFD dataset, it outperformed the runner-up by 15.7%, demonstrating its superior ability to filter out complex reconstruction artifacts.
  • Generalization: SUMI-IFL showed strong performance on unseen data, improving upon existing methods by 9.7% on the NIST16 benchmark.
  • Ablation Studies: Removing the sufficiency and minimality constraints resulted in a 9.8% drop in F1 score on DEFACTO-12, confirming that these information-theoretic constraints are the primary drivers of the model's success.

Conclusion

SUMI-IFL introduces a mathematically grounded approach to image forensics by leveraging Information Bottleneck theory. By balancing the need for comprehensive feature extraction with strict noise filtering, the framework achieves significant improvements in localization accuracy and robustness. Future research may focus on reducing the computational complexity of the multi-stream architecture or extending these principles to video-based forgery detection.

Summary updated

Abstract

Image forgery localization (IFL) is a crucial technique for preventing tampered image misuse and protecting social safety. However, due to the rapid development of image tampering technologies, extracting more comprehensive and accurate forgery clues remains an urgent challenge. To address these challenges, we introduce a novel information-theoretic IFL framework named SUMI-IFL that imposes sufficiency-view and minimality-view constraints on forgery feature representation. First, grounded in the theoretical analysis of mutual information, the sufficiency-view constraint is enforced on the feature extraction network to ensure that the latent forgery feature contains comprehensive forgery clues. Considering that forgery clues obtained from a single aspect alone may be incomplete, we construct the latent forgery feature by integrating several orthogonal individual image features. Second, based on the information bottleneck, the minimality-view constraint is imposed on the feature reasoning network to achieve an accurate and concise forgery feature representation that counters the interference of task-unrelated features. Extensive experiments show the superior performance of SUMI-IFL to existing state-of-the-art methods, not only on in-dataset comparisons but also on cross-dataset comparisons.
BibTeX
@inproceedings{sheng2025sumi,
  author = {Ziqi Sheng and Wei Lu and Xiangyang Luo and Jiantao Zhou and Xiaochun Cao},
  title = {{SUMI-IFL: An Information-Theoretic Framework for Image Forgery Localization with Sufficiency and Minimality Constraints}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i1.32054},
}
Figure