A camera signature proves that nothing changed after capture. It does not prove the camera was pointed at something real. A trusted camera will sign a photo of a printed photograph, a video replayed on a tablet held up to the lens, or a real-time deepfake injected into the stream. It signs each of these as faithfully as it signs an honest scene. Security researchers call this the "authenticated-but-wrong" data problem. Hardware enclaves (TEE-based off-chain data collection) and zero-knowledge proofs (Public Accountability vs. Secret Laws) move the trust boundary right up to the sensor, and then stop there. This proposal turns a well-known weakness of computer vision into a check that works at that boundary.
The weakness is that inverse rendering is ill-posed. Inverse rendering means recovering shape, surface color, and lighting from pixels. One image can be explained by endlessly many combinations of those three: a dark surface under bright light looks the same as a bright surface under dim light. So a single image can never prove it came from a real scene. The situation changes when the device itself lights the scene. Suppose the trusted camera flashes a secret, randomly chosen sequence of lighting patterns from its screen or LEDs at known positions. A real scene must then respond in a narrowly constrained way. The response adds up linearly across lights, and it follows one fixed shape and color under all of them. The core idea is to test that constraint inside the enclave, not to solve inverse rendering fully. A print, a replay screen, or a forger that relights too late will fail the test. The test result is then signed together with the frames.
Core Claim
A lightweight physical-consistency test can supply a veracity signal that a learned classifier cannot. The test asks whether a short burst of frames captured under a secret sequence of lighting patterns is explained by one fixed scene responding linearly to the lights the device actually emitted. The test combines three ingredients: secret and fast switching of the code, several spatially separated emitters, and a check that one geometry and one surface color explain every pattern.
Let $I_t$ be the frame at time $t$, and let $c_{t,k}\in\{0,1\}$ be the secret on/off state of emitter $k$ during that frame. Because light adds up, a static real scene satisfies
$$I_t = A + \sum_{k=1}^{K} c_{t,k}\, T_k + \epsilon_t,$$
where $A$ is the ambient image, $T_k$ is the image the scene would produce under emitter $k$ alone, and $\epsilon_t$ is sensor noise. The per-emitter images $T_k$ must also agree with one shape. Under a near-field diffuse model this reads $T_k(p)\approx \rho(p)\,\max(0,\mathbf{n}(p)\cdot \mathbf{l}_k(p))/r_k(p)^2$, where $\rho$ is the surface color at pixel $p$, $\mathbf{n}$ is the surface normal, $\mathbf{l}_k$ is the known unit direction to emitter $k$, and $r_k$ is the distance to it. Specular highlights are allowed as a separate, sparse residual term.
The veracity score combines two residuals. The first is the linear-fit residual, which exposes timing and replay failures. The second is the shape-consistency residual, which exposes flat or wrongly shaded responses. Both are cheap least-squares computations, well within the budget of an enclave.
What would be new is the pairing of two properties. The first is that the challenge code stays sealed inside the attesting device and is committed to the signed record. The second is that the decision rests on an explicit light-transport model rather than learned features. This makes the check auditable by anyone who holds the revealed code and the frames, and it gives an attacker a precise physical task to fail.
Relation to Prior Work
The closest work is Gotcha, which motivates this proposal and is the main competitor. Gotcha challenges a live caller with phone-flash and screen-brightness changes, occlusions, and head motion. It then classifies the stream with a 3D-ResNet18 trained with a contrastive loss, reaching an area under the ROC curve (AUC) of 80.1% against 88.62% for human judges. Its illumination challenges are chosen by the verifier but are neither calibrated nor secret-coded per frame. It never checks light transport explicitly, and it never binds the challenge to a signed record. The present proposal keeps Gotcha's insight that the verifier should force physics onto the data provider. It replaces the learned judgment with a transport-consistency test, and it moves the challenge inside the trusted capture device.
CASIA-SURF supplies the other reference point. It shows that adding depth and infrared hardware defeats 2D print attacks, reporting 96.7% true positive rate at a false positive rate of $10^{-2}$ and an average classification error rate of 2.4%. It is passive and needs extra sensors, and nothing in it ties a recording to the moment of capture. The enclave work (TEE-based collection) and the reputation surveys (blockchain for cyber-physical systems, trust and reputation for IoT) define the setting. Those systems either trust whatever photons hit the sensor or score sensors from peer agreement, which fails when a sensor is isolated and has no peers.
On the vision side, the method borrows from the Multi-Illumination dataset. That paper uses the fact that radiance adds up across 25 bounce-flash directions to make material and lighting recoverable. It also borrows the classical photometric observation that several known light directions pin down normals and surface color. Several papers constrain the design by showing that ambiguity from a single image cannot be removed without strong priors: SfSNet, InverseRenderNet with 37.7° normal error in the wild, and the baked-in artifacts documented in PANDORA and Noise Guided Splatting. The verifier therefore never tries to decompose a single frame. Heavy solvers such as Inverse Path Tracing and PANDORA's polarimetric fields recover far more than is needed and cost minutes, not milliseconds. The proposal deliberately asks a weaker question that can be answered cheaply.
Research Program
The work has four parts.
Capture protocol. The hardware is a phone or a small camera module whose enclave controls both the image sensor and four to eight emitters. The emitters are screen quadrants or LEDs at calibrated offsets from the lens. Each attestation window lasts roughly half a second. During it, the enclave draws a code matrix $c$ from a keyed random generator and switches emitter states every frame. It records the frames, fits the linear model and the shape-consistency model, and outputs a signed record containing four things: a commitment to the code, hashes of the frames, both residuals, and a coarse normal and color map. Revealing the code later lets any auditor recompute the score. This fits the pattern of storing bulky data off the ledger and only a verifiable score on it, as the cyber-physical systems survey recommends.
Attacker hierarchy. The analysis frames the attacker's problem as prediction under latency. An attacker who controls a screen in front of the lens, or who injects frames between sensor and enclave, sees each code change only after it happens, with some delay $d$. If $d$ is at least the switching period $\tau$, each forged frame reflects the previous code, and the linear residual should grow in proportion to the fraction of mismatched frames. If $d<\tau$, the attacker must still render shading that differs by direction and matches the true geometry. A global brightness change fails the shape-consistency term. Five attackers are built, in increasing strength: a static print, an unmodulated replay screen, a replay screen that brightens globally after detecting a flash, a relighting forger that uses a network trained on multi-illumination data to render direction-specific shading with measured latency, and a physical 3D replica painted to match the real object's color.
Non-ideal scenes. Real scenes are not ideal. Shiny and metallic surfaces break the diffuse shape model; PANDORA documents exactly this failure for polarimetric models on conductors. Motion breaks the linear model, and strong ambient light shrinks the signal from the emitters. The program handles these in three ways. Specular reflections are modeled as a sparse residual. Motion is handled by fitting the model only on small aligned patches. Scenes where the emitter signal is too weak for any decision are detected and abstained on, rather than guessed about. Abstention is reported as a separate outcome.
Interface to the integrity stack. The two residuals become a calibrated veracity score. A reputation system of the kind described in trust and reputation for IoT can consume that score as direct evidence, which gives isolated optical sensors a check that does not depend on peers.
Baselines
The naive reference point is signing alone. An enclave camera signs every frame and accepts everything, so its detection rate against presentation and injection attacks is zero by construction. It shows what veracity checking adds on top of the integrity guarantee the TEE collection work already provides.
The strongest existing method on this exact question is Gotcha's learned challenge-response detector (Gotcha): a 3D-ResNet18 with a contrastive loss that reports 80.1% AUC on the GOTCHA benchmark, where passive detectors (SBI and FTCN) reach only 50-53%. For a fair comparison, that architecture will be retrained on the same newly collected coded-illumination data. It will receive the same frames and, in one variant, the code sequence as an extra input channel. It will get the same hyperparameter-search budget as the proposed calibration step, and it will be evaluated on the same held-out attack families. To beat it, the proposed test must match or exceed its AUC on seen attacks. It must also clearly exceed it on attack families held out from training, especially the adaptive relighting forger, which is where a physics test should generalize and a classifier should not.
The multimodal passive baseline is CASIA-SURF's squeeze-and-excitation fusion (CASIA-SURF), which reports 96.7% true positive rate at a false positive rate of $10^{-2}$ on its own test set. It will be run with a consumer depth-and-infrared module on the face subset of the new data. The proposed test beats it if it reaches comparable true positive rates on print and replay attacks with RGB and emitters alone, and if it catches the painted 3D replica and injection attacks, which depth alone should not flag.
Two ablations isolate the key ingredients. The first is a temporal-correlation-only test: correlate the brightness of each patch with the secret code, with no shape-consistency term. The second uses the full model but with a publicly predictable code, for example a fixed pattern. If the temporal-only test performs as well as the full test against the adaptive relighting forger, the geometric part of the idea is unnecessary. If the fixed code performs as well as the secret one, secrecy is doing no work.
Why not just…?
Why not just train a better classifier on illumination challenges, as Gotcha does? This is the real threat. A large enough network trained on coded captures might learn the linear and shape constraints implicitly. What argues against relying on it is generalization, not accuracy on seen attacks: Gotcha's own results show learned detectors falling to 50-53% AUC when moved off their training distribution. A classifier's decision also cannot be audited from the signed record. The head-to-head comparison on held-out attack families settles this. If the retrained classifier matches the physics test there, the honest conclusion is that the contribution reduces to the capture protocol, and the test serves mainly as an interpretable residual. A hybrid that feeds both residuals to the classifier is included for the same reason.
Why not just correlate frame brightness with the secret code? For a static print or an unmodulated replay, this is probably enough, and the ablation will show it. It fails, by design, against an attacker who notices each flash and brightens the whole forged frame. That attacker satisfies temporal correlation but produces the wrong pattern of shading across directions. The adaptive-attacker experiments measure exactly this gap. If it does not appear, the geometric term will be dropped.
Why not just add a depth sensor? Depth defeats flat spoofs, as CASIA-SURF and radiometric RGB-D decomposition show. It says nothing about surface color, so a painted 3D replica passes. It also does nothing to show that frames were captured at a particular moment rather than injected from a prior recording. Emitters already exist on every phone, so the coded-light test costs no new hardware. The painted-replica and injection attacks are included specifically to test whether this advantage is real.
Why not just run a full inverse renderer such as Inverse Path Tracing or PANDORA? Those methods recover complete material and lighting maps, reaching 0.006 rendering loss in Inverse Path Tracing and 3.91° normal error in PANDORA. They do so with minutes of optimization, and PANDORA needs a polarization camera and assumes non-metallic materials. Verification only needs to reject scenes that no single fixed scene could explain, a much weaker question than full recovery. One experiment runs a heavy solver offline as an upper bound, to measure how much accuracy the cheap test gives up.
Proposed Experiments
The first experiment is a simulation on the Multi-Illumination dataset. Its 1,016 HDR scenes under 25 known flash directions allow coded captures to be synthesized exactly by summing single-light images under random codes. Forgeries are generated in four ways. The first uses wrong or delayed codes. The second substitutes a flat display plane. The third applies a global brightness change. The fourth uses relit images from a U-Net trained on held-out scenes; that paper's lighting estimator reaches 26.6° mean angular error against 68.5° for Gardner et al., and no reported relighting error was found on Lacuna. For each forgery type, the experiment measures AUC and the false-rejection rate on honest scenes as functions of the number of emitters, the code length, and the attacker's latency. It also stratifies results by material masks, to measure how often shiny surfaces cause false rejections.
The second experiment uses newly collected hardware captures: phones with enclave-controlled screen quadrants and LEDs, recording faces and, for supply-chain-style records, consumer objects. Honest captures span varied ambient light and mild motion. They are paired with the five physical and digital attackers, with measured latencies.
The third experiment uses the GOTCHA benchmark (Gotcha), which lacks calibrated emitter geometry. Only the temporal portion of the test can run there, as an out-of-domain sanity check against the 80.1% AUC reference.
Two checks decide the project. The first is whether the shape-consistency residual separates the adaptive relighting forger from honest captures at a low false-rejection rate, in a regime where both the temporal-only ablation and the retrained Gotcha-style classifier fail on held-out attacks. The second is whether detection holds up when attacker latency falls below one frame period. That would show the geometric constraint, not the timing alone, is doing the work. If both checks pass, secret coded light becomes a practical physical veracity signal that can be committed to a ledger. If only the first passes, the result still narrows what a cheap RGB-only check can guarantee. If neither passes, the proposal reduces to a better data-collection protocol for learned detectors, and the physics-based claim should be abandoned.
References
- Extending On-Chain Trust to Off-Chain - Trustworthy Blockchain Data Collection Using Trusted Execution Environment (TEE)
- Public Accountability vs. Secret Laws: Can They Coexist?
- Gotcha: A Challenge-Response System for Real-Time Deepfake Detection
- A Dataset and Benchmark for Large-Scale Multi-Modal Face Anti-Spoofing
- Challenges in Designing Blockchain for Cyber-Physical Systems
- Trust and Reputation Management for Blockchain-enabled IoT
- A Dataset of Multi-Illumination Images in the Wild
- SfSNet: Learning Shape, Reflectance and Illuminance of Faces 'in the Wild'
- InverseRenderNet: Learning Single Image Inverse Rendering
- PANDORA: Polarization-Aided Neural Decomposition of Radiance
- Fix False Transparency by Noise Guided Splatting
- Inverse Path Tracing for Joint Material and Lighting Estimation
- Radiometric Scene Decomposition: Scene Reflectance, Illumination, and Geometry from RGB-D Images