Detectors of AI-generated images break in a predictable way when images are compressed, resized or blurred. The tiny high-frequency traces that generators leave behind are erased, and the detector does not report "I can't tell." It reports "real." We propose to fix this at the step where a detector combines evidence across an image. We split the image into patches. Each patch gets a fake-versus-real score and, separately, a reliability score that says whether that patch still holds usable forensic evidence. Unreliable patches abstain instead of voting "real." The reliability score is computed from simple visual cues of each patch, such as local edge sharpness, texture richness, residual mid-frequency energy and compression blockiness. It is deliberately not taken from the classifier's own confidence, because a classifier whose evidence has been erased is often confidently wrong.
The two-sentence version: post-processing turns "no evidence" into "evidence of real," and averaging patch predictions bakes that error into the image-level decision. If a separate, cue-based estimate decides which patches are allowed to vote, laundered fakes should keep their fake verdict whenever even a few patches retain traces, and genuine photos should not be pushed toward "fake."
Core Claim
The asymmetry is documented. Dissect and Prune shows a detector's fake accuracy on FLUX images falling from 95.9% to 12.2% after post-processing. Its fix, DEAR, prunes the final-layer feature channels whose activations differ most between fake and real regions, raising fake accuracy to 89.5%. DEAR treats the problem as a global property of the feature space. We treat it as a property of where in the image evidence survives, and we leave the network's features untouched. Degradation is spatially uneven, and so is the content that carries generator traces. Flat sky loses everything under JPEG compression, while dense texture and sharp boundaries keep mid-frequency structure.
Our claim is that reliability-weighted patch aggregation with abstention reduces post-processing asymmetry more than three alternatives: uniform averaging, confidence weighting, and learned attention pooling. It should also complement both feature-level pruning and robust foundation backbones.
What Prior Work Gives Us and Where It Stops
Patch aggregation in forensics is established but unweighted.
- Camera-model identification from 64×64 patches uses majority voting (First Steps Toward Camera Model Identification With Convolutional Neural Networks). That work reports vulnerability to heavy post-processing.
- RPA (Scalable Black-Box Model Attribution for Images) tiles images into 256×256 patches, classifies raw pixels with a 6M-parameter CNN, and averages probabilities. It reaches 98.0% accuracy on DRAGON with augmentation-driven robustness.
These papers supply our patch pipeline. We change only the averaging step.
Outside forensics, [CLS] is Not Enough shows that filtering uninformative patches and fusing max-pooled patch evidence with a global score lifts zero-shot CLIP on MS COCO from 49.2% to 68.8% mAP. That is evidence that selective aggregation matters, but it was shown on clean images for object recognition, not under degradation.
The cue design borrows from multi-cue saliency, which combines several kinds of visual evidence into one importance map.
- The UFO model (Salient Region Detection by UFO) fuses uniqueness, focusness (edge sharpness) and objectness so that one failing cue can be covered by others.
- SCOUT uses element-wise products as a strict logical AND between cues. - Priority coding for video-telephony uses saturating functions before summation, so that weak cues cannot dilute a strong one.
These papers constrain our fusion rule. A pure product collapses when blur erases focusness. A pure sum lets uninformative cues dilute the rest. We therefore fuse cues with a generalized mean whose exponent is learned, and we test product and sum as ablations.
Two further lines of work set the robustness bar.
- Unveiling the Impact of Image Transformations on Deepfake Detection finds that a CLIP linear probe drops from 99.6% to 41.6% average accuracy across 12 transformations. DINOv2 holds 92.3%.
- Understanding Why Foundation Models Work for Diffusion-Generated Image Detection attributes the success of foundation models to low-to-mid frequency cues. It also shows that swapping in the lowest 5% of a real image's frequencies can flip a fake back to "real."
That result tells us that even robust backbones carry spatially uneven evidence, which aggregation can exploit.
Learning to Recognize Patch-Wise Consistency for Deepfake Detection and Patch-Discontinuity Mining for Generalized Deepfake Detection model relations between patches to find splicing. Neither addresses fully synthetic images, where every patch comes from the same source, and neither decides which patches deserve a vote.
Amplifying Discrepancies fuses macro and micro cues for localizing manipulated regions robustly. It produces a dense pixel map, not an image-level verdict built from abstaining units.
Research Program
Components. Each patch $i$ gets a fake log-odds score $s_i$ from a patch classifier. We use two backbones: an RPA-style raw-pixel CNN, and a linear head on frozen DINOv2 patch tokens.
Each patch also gets a reliability $r_i \in [0,1]$ from a small separate network. Its inputs are cue maps computed from the patch alone:
- local gradient sharpness (the focusness cue);
- texture entropy and Gram-statistic energy, following the global texture idea in Global Texture Enhancement for Fake Face Detection in the Wild;
- band-limited spectral energy in low, mid and high frequency bands;
- an 8×8 blockiness estimate that measures how strongly JPEG compression has acted.
Cue fusion. The cues are fused with a generalized mean, $$r_i = \sigma\Big(\big(\textstyle\sum_k w_k\, c_{ik}^{\,p}\big)^{1/p}\Big),$$ where $c_{ik}$ is cue $k$ at patch $i$, $w_k$ are learned non-negative weights, $\sigma$ is the logistic function, and $p$ is learned. As $p \to -\infty$ the mean approaches a minimum, which behaves like an AND gate. At $p = 1$ it is a weighted sum. The data decides where between these the fusion should sit.
Aggregation. The image score is a reliability-weighted log-odds plus a prior term: $$S = \frac{\sum_i r_i\, s_i}{\sum_i r_i + \lambda} + b,$$ where $\lambda$ is a learned constant. When few patches are reliable, $S$ shrinks toward the bias $b$, which is calibrated to the base rate, rather than toward "real."
Training the reliability network. The target is whether the frozen patch classifier gets that patch right after a randomly sampled degradation chain of JPEG, resizing and blur. Degradation chains are drawn from a pool that is disjoint from the evaluation chains. The cue inputs never see $s_i$. Keeping reliability independent of the classifier's confidence is the key design choice, and it is what the ablations isolate.
Analysis. Beyond the headline numbers, we will:
- measure how $r_i$ tracks surviving spectral energy;
- check whether reliable patches concentrate on textured, object-like regions, which would confirm the saliency intuition, or somewhere else;
- report the learned exponent $p$ at each severity level, to answer how cues should be fused under degradation.
Baselines
Uniform patch averaging. The naive reference is uniform probability averaging over the same patches and the same patch classifier, which is RPA's aggregation rule. Hard majority voting, as in Bondi et al., is a second naive variant. No result on binary real-versus-fake detection under post-processing is reported on Lacuna for this aggregation rule. The reported RPA figures (98.0% on DRAGON, 92.9% on OpenFake) are closed-set attribution accuracy. They are not comparable, and we will not present them as such. Beating this baseline means a higher fake accuracy on degraded fakes without losing real accuracy.
DEAR. The strongest existing method aimed at exactly our failure mode is Dissect and Prune. Its reported numbers are an average post-processed FLUX fake accuracy of 89.5%, and an AUC on LOKI of 78.5 against 74.6 for its base detector. We will:
- run DEAR on the same base detector, training data and post-processing protocol;
- reproduce its FLUX numbers before comparing;
- test our aggregator both instead of DEAR and stacked on a DEAR-pruned backbone.
Beating DEAR means exceeding its post-processed fake accuracy at equal or better real accuracy. Showing complementarity means the stacked version beating both components.
Frozen DINOv2 global linear probe. The strongest robustness baseline is a global linear probe on frozen DINOv2. Unveiling the Impact of Image Transformations reports 92.3% average accuracy across 12 transformations on COCOFake for this probe. Our DINOv2-patch variant uses the identical frozen backbone and training images. Beating it means a higher average transformed accuracy on COCOFake, especially at the most severe JPEG settings.
Ablations and tuning budget. The ablations isolate the key ingredient:
- confidence or entropy weighting, where $r_i$ comes from the classifier's own $|s_i|$;
- learned attention pooling trained end-to-end on the same augmented data, in the multiple-instance-learning style;
- top-K pooling, as in CapsDeMM;
- our aggregator with product-only and sum-only cue fusion;
- our aggregator with the prior term removed ($\lambda=0$).
All aggregation variants get the same hyperparameter-search budget and the same held-out degradation pool for tuning.
Why not just…?
Why not just weight patches by the classifier's own confidence? This is the cheapest alternative, and it targets the wrong signal. The asymmetry reported in Dissect and Prune means laundered fakes are pushed toward confident "real." Confidence weighting would then up-weight exactly the misleading patches. The confidence-weighting ablation tests this directly. If it matches our cue-based reliability, the independence argument fails, and we would report the simpler method.
Why not just learn attention pooling end-to-end? This is the real threat. A learned attention head could discover reliability on its own. The risk is that, trained on the same augmented data, it learns the same degradation shortcuts as the classifier. It also has no abstention prior. We run it with identical data and budget, then evaluate it on degradation chains held out from training. If it matches us on held-out chains, the contribution shrinks to the abstention prior and the interpretable cue analysis, and we will say so.
Why not just use a robust backbone like DINOv2 and skip patches? DINOv2 already holds 92.3% average accuracy on COCOFake. But Understanding Why Foundation Models Work for Diffusion-Generated Image Detection shows that its evidence can be flipped by low-frequency manipulation. A single global decision cannot tell an erased region from a genuinely real one. If our DINOv2-patch variant gives no gain over the global probe at the highest severities, aggregation adds nothing on top of a robust backbone. That finding would itself be useful.
Why not just add more augmentation? RPA already trains with heavy augmentation, and our uniform-averaging baseline inherits it. Unveiling the Impact of Image Transformations and the FLUX results show that asymmetry persists under unseen post-processing even when training data is diverse. Because every arm in our comparison shares the same augmentation, any gain we measure comes from aggregation.
Proposed Experiments
Training data. Training follows the GenImage protocol of training on Stable Diffusion 1.4 (GenImage). There, a ResNet-50 falls from 99.9% to 54.9% accuracy on Midjourney, and downsampling to 64×64 drops it to about 57%.
Evaluation sets.
- GenImage across generators, at graded JPEG quality, resizing and blur;
- COCOFake under its 12 transformations;
- the FLUX post-processing protocol from DEAR;
- in-the-wild sets: LOKI, plus Chameleon and WildRF, for which no reported result was found on Lacuna for either set;
- RAID, as a secondary stress test only, since our method does not target adversarial noise.
Metrics. We report fake accuracy and real accuracy separately, because asymmetry is the point. We also report balanced accuracy, AUC, and an asymmetry gap defined as the difference between real and fake accuracy on post-processed images.
Decisive checks. Three results settle the proposal:
- Cue-based reliability must beat both confidence weighting and learned attention on held-out degradation chains. If it does not, the core claim is false.
- The asymmetry gap must shrink as severity increases relative to uniform averaging, while real accuracy on clean images stays within one point.
- On images with spatially uneven damage, created by degrading half of each image while leaving the other half clean, our aggregator should recover nearly clean-image fake accuracy where averaging does not. This test directly checks whether abstention, rather than a better classifier, is doing the work.
References
- Dissect and Prune: Enhancing Robustness in AI-Generated Image Detection
- First Steps Toward Camera Model Identification With Convolutional Neural Networks
- Scalable Black-Box Model Attribution for Images
- [CLS] is Not Enough: Multi-Label Recognition via Patch-Level Inference and Adaptive Aggregation
- Salient Region Detection by UFO: Uniqueness, Focusness and Objectness
- SCOUT: Self-Aware Discriminant Counterfactual Explanations
- Priority coding for video-telephony applications based on visual attention
- Unveiling the Impact of Image Transformations on Deepfake Detection: An Experimental Analysis
- Learning Self-Consistency for Deepfake Detection
- Patch-Discontinuity Mining for Generalized Deepfake Detection
- Amplifying Discrepancies: Exploiting Macro and Micro Inconsistencies for Image Manipulation Localization
- Global Texture Enhancement for Fake Face Detection in the Wild
- CapsDeMM: Capsule Network for Detection of Munro's Microabscess in Skin Biopsy Images
- GenImage: A Million-Scale Benchmark for Detecting AI-Generated Image