The Courtroom Trial of Pixels: Robust Image Manipulation Localization via Adversarial Evidence and Reinforcement Learning Judgment

Songlin Li, Zhiqing Guo, Dan Ma, Changtao Miao, Gaobo YangarXiv 2026 · arXiv · OpenReview

TL;DR

This paper introduces a novel "courtroom-style" framework for image manipulation localization (IML) that treats the task as a debate between "prosecution" (manipulation evidence) and "defense" (authenticity evidence), culminating in a "judge's" ruling. This adversarial approach, enhanced by reinforcement learning for re-reasoning in uncertain regions, significantly improves robustness and accuracy, especially against subtle manipulations and post-processing artifacts.

What Problem Does This Paper Address?

Existing image manipulation localization (IML) methods often struggle when manipulation traces are subtle, degraded by post-processing (like JPEG compression or resizing), or when distinguishing between genuine and manipulated regions becomes ambiguous. Many approaches primarily focus on detecting manipulation artifacts, treating authenticity cues as secondary. This can lead to models that are overconfident in their incorrect predictions and lack the ability to explicitly weigh conflicting evidence. The core problem is the need for a more robust IML system that can effectively handle uncertainty and adversarial conditions by explicitly modeling and adjudicating between evidence of manipulation and evidence of authenticity.

Methods

The proposed framework adopts a courtroom analogy with three main components:

  • Prosecution and Defense Streams: A dual-hypothesis segmentation architecture is built on a shared multi-scale encoder. One stream acts as the "prosecution," identifying manipulated regions, while the other acts as the "defense," identifying authentic regions.
  • Dynamic Debate Mechanism: This mechanism facilitates interaction between the prosecution and defense streams. It uses bidirectional cross-attention and a "push-pull" update to suppress interference in consensus areas and amplify opposition in disputed regions. This helps refine the evidence from each stream, making them more distinguishable and less prone to misguidance. Edge priors are also incorporated to guide this process.
  • Reinforcement Learning (RL) Judge Model: This model acts as the "judge," taking the evidence from both streams and the original image to make a final decision. It employs an Actor-Critic RL approach to perform strategic re-inference and refinement on uncertain regions. The judge is trained using an advantage-based reward (relative improvement in Soft-IoU) and a soft-IoU objective. Reliability is further calibrated using entropy and cross-hypothesis consistency.

Datasets

The paper evaluates its method on several standard IML datasets:

  • CASIAv2 [19]
  • Coverage [20]
  • NIST16 [21]
  • CASIAv1 [19]
  • Columbia [22]
  • Korus [23]
  • DSO [24]
  • IMD2020 [25]

The paper also includes robustness evaluations on images processed by social media platforms like Facebook, Weibo, WeChat, and WhatsApp.

Key Insights and Ideas

  • Courtroom Analogy for IML: Framing IML as a debate between "prosecution" and "defense" provides a powerful conceptual model for handling conflicting evidence and uncertainty. This adversarial approach forces a more rigorous evaluation of both manipulation and authenticity cues.
  • Dynamic Debate for Evidence Refinement: The dynamic debate mechanism is crucial for resolving ambiguities. By allowing streams to interact and suppress conflicting signals while amplifying reliable ones, it creates more discriminative evidence for the judge.
  • RL Judge for Adaptive Re-Reasoning: The RL-based judge is not just a passive fusion module. It actively learns to identify and re-process uncertain regions, mimicking a human judge's ability to scrutinize difficult cases. This adaptive re-inference is key to robustness.
  • Reliability Calibration is Essential: Simply predicting manipulation is not enough. The paper emphasizes the importance of calibrating the model's confidence and reliability, especially in ambiguous or noisy scenarios, to ensure trustworthy predictions.

Key Results

  • State-of-the-Art Performance: The proposed method achieves superior average F1 scores on both in-distribution (ID) and out-of-distribution (OOD) settings compared to existing state-of-the-art (SOTA) IML methods. For instance, it achieved an average F1 score of 0.526 on ID datasets, outperforming the runner-up Mesorch (0.486).
  • Robustness: The model demonstrates exceptional robustness against various post-processing operations (JPEG compression, social media transformations) and common image corruptions (Gaussian noise, blur), achieving an average OOD F1 score of 0.401.
  • Ablation Studies: Removing key components like the dynamic debate mechanism or the RL judge significantly degrades performance, highlighting their critical contributions to the overall effectiveness and robustness.

Limitations and Future Work

  • Complexity and Computational Cost: The proposed courtroom-style framework is more complex than conventional single-stream IML methods, leading to higher training and optimization costs. This could be a barrier for deployment in resource-constrained environments.
  • Patch-Based Decision Making: The judge's current decision-making is primarily patch-based. This might limit its ability to capture global consistency for highly irregular manipulations or complex scenarios requiring long-range semantic dependencies.
  • Future Directions: The authors plan to explore more lightweight adjudication frameworks by simplifying modules or replacing parts of the RL process. They also aim to investigate hierarchical or adaptive-granularity adjudication mechanisms to better incorporate global semantic consistency and improve localization accuracy around challenging boundaries.

Summary updated

Abstract

Although some existing image manipulation localization (IML) methods incorporate authenticity-related supervision, this information is typically utilized merely as an auxiliary training signal to enhance the model's sensitivity to manipulation artifacts, rather than being explicitly modeled as localization evidence opposing the manipulated regions. Consequently, when manipulation traces are subtle or degraded by post-processing and noise, these methods struggle to explicitly compare manipulated and authentic evidence, resulting in unreliable predictions in ambiguous areas. To address these issues, we propose a courtroom-style adjudication framework that regards IML task as the confrontation of evidence followed by judgment. The framework comprises a prosecution stream, a defense stream, and a judge model. We first build a dual-hypothesis segmentation architecture on a shared multi-scale encoder, in which the prosecution stream asserts manipulation and the defense stream asserts authenticity. Guided by edge priors, it produces evidence for manipulated and authentic regions through cascaded multi-level fusion, bidirectional disagreement suppression, and dynamic debate refinement. We further develop a reinforcement learning judge model that performs strategic re-inference and refinement on uncertain regions, yielding a manipulated-region mask. The judge model is trained with advantage-based rewards and a soft-IoU objective, and reliability is calibrated via entropy and cross-hypothesis consistency. Experimental results show that our model achieves superior average performance compared with SOTA IML methods.
BibTeX
@misc{li2026courtroom,
  author = {Songlin Li and Zhiqing Guo and Dan Ma and Changtao Miao and Gaobo Yang},
  title = {{The Courtroom Trial of Pixels: Robust Image Manipulation Localization via Adversarial Evidence and Reinforcement Learning Judgment}},
  year = {2026},
  eprint = {2604.14703},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2604.14703},
}
Figure