Detecting AI-Generated Images Under Encryption with Second-Order Color Statistics

Suppose someone wants to know whether a photo is AI-generated but cannot show it to the company that runs the detector. The photo might be a leaked document, a medical image, or a source's picture. The company, in turn, will not ship its detector, because a released detector is easy to attack and to train generators against. Fully homomorphic encryption (FHE) addresses both concerns: it is a way to compute on encrypted numbers so the server never sees the image and the client never sees the model. The obstacle is cost. Encrypted evaluation of an ordinary image classifier has been measured at roughly 549 seconds and about 56 GB of traffic for one image (Haralampieva et al.). The main drivers are two things. Nonlinear layers must be approximated by polynomials. Deep multiplication chains force "bootstrapping", an expensive procedure that refreshes the noise inside ciphertexts (Wen et al.).

Our claim is that synthetic-image detection is unusually well suited to FHE, because its strongest known signal is nearly polynomial already. Generators leave their clearest traces in chrominance, the color-difference channels of a luma/chroma decomposition, and not in brightness. That decomposition is a linear map of RGB. The traces show up in how neighboring high-frequency residuals co-vary, and a residual is also linear. A detector built from averages of products of chrominance residuals is therefore a degree-2 polynomial in the pixels. It needs one or two ciphertext multiplications and no bootstrapping at all. We propose to build this detector, measure how much accuracy it gives up relative to non-polynomial color forensics, and run it end-to-end under CKKS, an FHE scheme for approximate real-valued arithmetic.

Core Claim

The forensic evidence behind color-based detection sits largely in low-order moments of chrominance residuals. The following findings from prior work motivate this claim.

  • Li et al. apply first-order differences to HSV and YCbCr channels and classify co-occurrence matrices of those residuals. They report 99.74% matched accuracy and 81.11% face-to-bedroom transfer. They also find that the real-versus-fake chi-square disparity is 0.035 in $Cb$ against only 0.004 in $Y$.
  • Chandrasegaran et al. find that about 85% of the most transferable features in a ResNet-50 detector are color-conditional. Converting inputs to grayscale lowers the median fake probability by over 89%.
  • Jia et al. show that a 1.4M-parameter network on a color-difference map reaches 98.2% accuracy on GAN images and 97.5% on the Ojha diffusion benchmark. This suggests the discriminative computation can be shallow once the right representation is chosen.

None of these methods is FHE-friendly. Co-occurrence counting, HSV conversion, 4-bit quantization, and the min/max operations behind dark- and bright-channel priors (Guo et al.) are all comparisons or step functions. Such operations need deep polynomial approximations.

A co-occurrence matrix records the joint distribution of neighboring residual values. Its second moments are just products of neighboring residuals averaged over the image, and they are polynomial. Concretely, let $x$ be the image, $A$ the fixed RGB-to-YCbCr matrix, and $h_a$ a small residual filter. The residual maps are $r_a = h_a * (Ax)$. The features are

$$\phi_{ab\delta} = \frac{1}{|P|}\sum_{p \in P} r_a(p)\, r_b(p+\delta),$$

where $P$ is the set of pixel positions and $\delta$ ranges over a few small spatial offsets. The detector score is $s = w^\top \phi + c$, a degree-2 polynomial in $x$. The filters $h_a$ and the weights $w$ are learned. An optional variant, $s = \phi^\top M \phi + w^\top \phi + c$ with learned matrix $M$, reaches degree 4 at one further multiplicative level.

The new question is empirical: does replacing co-occurrence histograms by learned second-order moments preserve the chrominance signal? If it does, private detection costs a bootstrapping-free leveled computation. That is a different regime from the multi-minute, multi-gigabyte deep-CNN numbers in the literature.

Relation to Prior Work

The research file finds no prior work running color or luminance forensics under FHE. The closest pieces each supply one ingredient.

Haralampieva et al. benchmark private inference for generic image classifiers and show that HE-Transformer preserves plaintext accuracy while paying 549 s and 56 GB. We take their evaluation protocol and their cost measurements as the reference. We differ in choosing a task whose signal allows a very shallow circuit.

Three works supply methods for making networks polynomial:

  • CryptoNets uses square activations and batch packing. We reuse its packing idea.
  • SAFENet keeps expensive nonlinearities only on critical channels.
  • PolyNeXt removes activations entirely from a MetaFormer-style backbone, reaching 82.7% ImageNet top-1 when fully polynomial.

All three target semantic recognition and approximate a deep network after the fact. PolyNeXt also depends on depth, since an $L$-layer network represents a polynomial of degree $2^L$, and that is exactly what leveled FHE penalizes. We instead start from a forensic statistic that is low-degree by construction. SecureNets states the trade-off we exploit: low degree is cheap but may lose accuracy.

The color-space choice is constrained by Gap 1 in the file. HSV needs division and branching, so we restrict ourselves to affine decompositions: YCbCr and the $R-G$ and $B-G$ planes used by Zhang et al..

One cited result cuts against us. Grayscale collapse (Chandrasegaran et al.) implies that a chrominance-only detector may be easy to evade by color grading. We treat this as a robustness test we must report, not a footnote.

Research Program

The work proceeds in three stages.

Stage 1: plaintext accuracy. We characterize the accuracy of the moment detector without encryption. Filters are initialized to first-order differences, matching Li et al., and then learned. We sweep the number of filters, the number of offsets, and the channel set (luma only, chroma only, both). We compare a linear head (degree 2) with a quadratic head (degree 4). Every variant is trained with the encryption-time arithmetic simulated: fixed-point scaling plus injected CKKS-magnitude noise. This addresses the file's Gap 5 concern that rounding could erase disparities as small as 0.035.

Stage 2: CKKS implementation. We implement the detector in CKKS. As an example of the target configuration, a 128×128 crop has 16,384 pixels per channel, which fills the slots of one ring-dimension-$2^{15}$ ciphertext per channel. Filtering and the color transform become slot rotations plus plaintext multiplications. The products $r_a(p)r_b(p+\delta)$ are the only ciphertext-ciphertext multiplications. Spatial averaging uses a logarithmic number of rotate-and-add steps. The server returns only the encrypted score.

Stage 3: cost and threat model. We measure latency, ciphertext and key sizes, and throughput under the batch packing of CryptoNets. We state the threat model plainly. The model weights enter as encoded plaintexts, but query access still permits black-box attacks; FHE protects the image and hides the weights from white-box inspection, and nothing more.

Baselines

The naive reference point has two parts. The first is chance accuracy (50%). The second is the generic encrypted classifier cost reported by Haralampieva et al.: 548.88 s and about 56 GB per image for HE-Transformer on their Malaria Model Z. That is a different task, so we treat it as an order-of-magnitude anchor, not a head-to-head comparison.

The strongest plaintext competitors are three:

  • Li et al.'s chrominance co-occurrence detector. It is the non-polynomial version of our own statistic, reporting 99.74% matched accuracy. Beating it is not required; matching it within a few points is the goal.
  • CoD with its 1.4M-parameter ResNet. Jia et al. report 97.5% on the Ojha diffusion set and 98.2% on Forensic Synthetics GANs.
  • GenImage's ResNet-50 and Swin-T. ResNet-50 gets 99.9% matched on SD V1.4 and 54.9% on Midjourney. Swin-T gets 74.8% average cross-generator accuracy (GenImage).

We retrain all of these ourselves on identical training splits and image crops, with equal hyperparameter-search budgets. The published numbers serve as sanity checks that our retraining is faithful. No reported result was found on Lacuna for any AI-image detector evaluated under FHE, so there is no encrypted forensic number to beat.

The strongest encrypted-regime competitor is a generic polynomial network trained on raw RGB crops for the same task. We test two versions, a PolyNeXt-style fully polynomial backbone and a CryptoNets-style square-activation CNN, each at matched multiplicative depth budgets of 2, 4, and 8 levels. This asks whether the forensic front end matters or whether any shallow polynomial network learns the same thing.

Three ablations isolate the key ingredient:

  • luma-only moments versus chroma moments;
  • fixed first-order difference filters versus learned filters;
  • degree-2 versus degree-4 heads.

Beating the generic polynomial network means higher cross-generator accuracy on GenImage at equal or lower multiplicative depth. The plaintext-to-encrypted gap should be under one point of accuracy.

Why not just…?

Why not just run the standard detector in a trusted enclave? Haralampieva et al. measured 0.14 s for TF-Trusted on SGX. No FHE design will approach that latency, and we do not claim it will. The enclave's guarantee depends on trusting a hardware vendor and on the enclave's limited protected memory. FHE's guarantee is cryptographic. The honest framing is that this work matters only for users who reject hardware trust. We report our latency next to the enclave number so readers can judge the price.

Why not just use secure multi-party computation with exact ReLUs? CrypTen ran Model Z in 0.25 s, but on a LAN, and it requires the client to stay online through many interactive rounds. Our protocol has one upload and one download. We will simulate WAN round-trip times for a CrypTen implementation of the CoD network and compare it with our non-interactive latency. If CrypTen still wins at realistic WAN delays, we will say so.

Why not just take any polynomial CNN, such as PolyNeXt, and encrypt it? This is the real threat. If a depth-matched generic polynomial network on RGB matches our moment detector, the forensic front end adds nothing. The depth-matched comparison in the baselines settles it. Our expectation, not a result, is as follows. At depth 2 to 4, a generic network cannot synthesize chrominance-residual correlations from raw pixels as reliably as a statistic that computes them directly, given that 85% of transferable features are color-conditional. If that expectation fails, we will pivot to reporting the generic network as the recommended design.

Why not just ship a grayscale detector to shrink ciphertexts threefold? The evidence against this already exists. Luma disparity is nearly ten times smaller than $Cb$ disparity (0.004 vs 0.035), and grayscale conversion cuts fake probability by over 89%. Our luma-only ablation re-tests this within our own detector.

Proposed Experiments

We train and evaluate on GenImage using the leave-one-generator-out protocol, with Midjourney and BigGAN as the hardest held-out generators. We also use the Ojha diffusion and Forensic Synthetics sets from Jia et al., and FakeART for cross-domain transfer. For degradation, we reproduce GenImage's 64×64 downsampling and add color-grading and saturation perturbations. For adversarial robustness we use RAID. Metrics are accuracy and AUC; encrypted-versus-plaintext score agreement; and per-image latency, amortized throughput, ciphertext traffic, and evaluation-key size.

Three checks decide the outcome.

  • Plaintext accuracy. The degree-2 chroma-moment detector should come within about 3 points of Li et al.'s co-occurrence detector on matched data and within about 5 points cross-generator. If it trails by more than 10 points cross-generator, the central claim is false.
  • Encrypted fidelity and cost. CKKS evaluation should reproduce the plaintext decision on essentially all test images with no bootstrapping. Its per-image traffic should be far below the tens of gigabytes reported for generic HE inference.
  • Robustness. Under color-grading perturbations, we measure how much of the chroma advantage survives. We then test whether color-jitter training recovers it without raising polynomial degree. A large unrecoverable drop would limit the method to settings where images arrive unedited, and we would report it as such.