One Class Anomaly Detection for Media Forensics

66 concepts49 papers1 proposal

The traditional approach to detecting digital forgeries—whether deepfake videos, synthetic voices, or spliced images—relies on binary classification. A neural network is trained on a balanced dataset containing thousands of authentic samples and thousands of fakes, learning a decision boundary that separates the two classes.

However, this paradigm faces a fundamental generalization wall. The space of potential forgeries is infinite and rapidly expanding. Every time a new generative architecture drops, the old decision boundary collapses because the model has overfitted to the specific artifacts of its training set. Furthermore, supervised models often exploit "shortcuts"—spurious dataset artifacts—rather than learning actual forgery traces, as exposed by Smeu et al. in Circumventing Shortcuts in Audio-visual Deepfake Detection Datasets with Unsupervised Learning. As Tian et al. note in Real Appearance Modeling for More General Deepfake Detection, the variety of methods for generating synthetic content is virtually limitless.

To build future-proof detectors, researchers are shifting the paradigm from pattern matching known fakes to one-class anomaly detection. Instead of trying to model the infinite variations of fakes, these systems learn a tight, comprehensive representation of what real media looks like. Anything that deviates from this learned manifold of reality is flagged as an anomaly.

The Geometry of Authenticity

To understand why one-class anomaly detection works, imagine a high-dimensional embedding space. Authentic media samples—constrained by the physics of real-world cameras, human anatomy, and digital recording pipelines—naturally cluster into a compact, low-variance manifold. Forgeries, regardless of how they were generated, introduce statistical inconsistencies that push them outside this authentic boundary.

One way to enforce this mathematically is through single-center optimization. Li et al. (Frequency-Aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detection) and Kong et al. (Enhancing General Face Forgery Detection via Vision Transformer with Low-Rank Adaptation) utilize a Single-Center Loss (SCL) to compress authentic samples into a tight hypersphere around a single central point. Because the model is heavily penalized if a pristine sample drifts away from the center, it masters the core invariants of real data. When a novel forgery is presented, it naturally falls far from the center, triggering an alert.

Alternatively, researchers apply classical one-class classifiers on top of deep embeddings. Cocchi et al. (Contrasting Deepfakes Diffusion via Contrastive Learning and Global-Local Similarities) use a One-Class Support Vector Machine (OC-SVM) to draw a tight boundary around real image embeddings, providing a robust defense against out-of-distribution generators like Midjourney or DALL-E 3. Similarly, Larue et al. (SeeABLE: Soft Discrepancies and Bounded Contrastive Learning for Exposing Deepfakes) map pristine faces modified with subtle, synthetic "soft discrepancies" to fixed geometric prototypes, forcing the network to learn the exact boundaries of the authentic manifold without overfitting to any single generator's fingerprint.

Reconstruction as a Forensic Filter

An elegant mechanism for authentic-only modeling is the autoencoder. If a network is trained only to compress and reconstruct pristine data, it becomes highly specialized in representing the statistical distributions of reality.

[Input Image] ---> [ Encoder ] ---> [ Bottleneck ] ---> [ Decoder ] ---> [Reconstructed Image]
                                                                                   |
                                                                         (Compare with Input)
                                                                                   |
                                                                         [Reconstruction Error]
                                                                         /                    \
                                                                (Low: Authentic)        (High: Anomaly)

When a forged image or video is passed through this filter, the bottleneck lacks the capacity to encode the anomalous artifacts. The network fails to reconstruct the tampered regions accurately, resulting in a high reconstruction error that exposes the forgery.

Khalid et al. implement this via a One-Class Variational Autoencoder in OC-FakeDect: Classifying Deepfakes Using One-class Variational Autoencoder, capturing the compact distribution of real facial features. To not only detect but also localize exactly where a face has been tampered with, Ma et al. (End-to-End Reconstruction-Classification Learning for Face Forgery Detection) and Hu et al. (Delocate: Detection and Localization for Deepfake Videos with Randomly-Located Tampered Traces) leverage masked autoencoders. By forcing the model to reconstruct missing patches of real faces, any forged region stands out due to its poor reconstruction fidelity.

This concept extends naturally to video forensics. D'Avino et al. (Autoencoder with recurrent neural networks for video forgery detection) use recurrent autoencoders to model a camera's unique statistical fingerprint over time, flagging green-screen compositions as temporal anomalies because the forged elements do not conform to the camera's baseline noise characteristics.

Grounding Reality in Physical and Behavioral Invariants

One-class models succeed because real-world media generation is bound by physical and biological constraints, whereas synthetic generation is not. By explicitly targeting these invariants, detectors can identify fakes without ever seeing them during training.

Camera and Sensor Physics

Real photographs carry invisible artifacts from physical camera sensors and hardware pipelines. Cozzolino et al. (Zero-Shot Detection of AI-Generated Images) introduce ZED (Zero-shot Entropy-based Detector), which uses a lossless image encoder to model natural pixel entropy, flagging synthetic images as "surprising" anomalies.

Li et al. (Identification of deep network generated images using disparities in color components) exploit noise residuals in chrominance channels that reflect physical camera sensor constraints. Taking this further, Zhong et al. (Self-Supervised AI-Generated Image Detection: A Camera Metadata Perspective) train models to predict EXIF metadata from image patches, anchoring the model's understanding in the intrinsic physics of photography.

Digital Processing Fingerprints

Localized tampering disrupts consistent processing traces. Bi et al. (Self-Supervised Image Local Forgery Detection by JPEG Compression Trace) and the authors of Comprint: Image Forgery Detection and Localization Using Compression Fingerprints train models exclusively on pristine data to learn the uniform profile of JPEG compression, exposing splices as local statistical inconsistencies. Similarly, Bammey et al. (An Adaptive Neural Network for Unsupervised Mosaic Consistency Analysis in Image Forensics) adapt a lightweight network to the specific demosaicing mosaic fingerprint of a single image to catch local edits.

Biological and Behavioral Invariants

Human speech and movement are constrained by vocal tract anatomy and behavioral rhythm. Villalba López et al. (Spoofing detection with DNN and one-class SVM for the ASVspoof 2015 challenge) and Wu et al. (Light Convolutional Neural Network with Feature Genuinization for Detection of Synthetic Speech Attacks) model the low-variance distribution of genuine human speech (such as magnitude and relative phase shifts) to detect synthetic audio.

For video, Liang et al. (SpeechForensics: Audio-Visual Speech Representation Learning for Face Forgery Detection) and the authors of Self-Supervised Video Forensics by Audio-Visual Anomaly Detection model the natural temporal synchronization and correlation between lip movements and audio streams. Because deepfakes often disrupt this behavioral manifold, they are easily flagged as temporal anomalies.

Broadening the One-Class Paradigm

The utility of authentic-only modeling extends beyond deepfakes into broader security contexts. For instance, Pianese et al. (Deepfake audio detection by speaker verification) reframe deepfake audio detection as a speaker verification task, requiring only the target speaker's authentic voice distribution to flag synthetic impersonations.

In physical security, the authors of Attacking Face Recognition with T-shirts: Database, Vulnerability Assessment and Detection and Liu et al. (Multi-Modal Face Anti-Spoofing Attack Detection Challenge at CVPR2019) demonstrate that framing face anti-spoofing as an anomaly detection task allows systems to catch novel presentation attacks (like printed T-shirts) by training exclusively on legitimate biometric samples.

Even in web security and intellectual property, the principle holds: Corona et al. (DeltaPhish: Detecting Phishing Webpages in Compromised Websites) suggest using one-class classification to model the legitimate characteristics of a specific web host to detect injected phishing pages, while Park et al. (DeepTaster: Adversarial Perturbation-Based Fingerprinting to Identify Proprietary Dataset Use in Deep Neural Networks) apply one-class classification to the spectral signatures of adversarial perturbations to verify if a proprietary dataset was stolen to train a model.

By anchoring the detection boundary to the finite, verifiable characteristics of authentic data, one-class anomaly detection breaks out of the cat-and-mouse cycle of supervised learning, offering a robust framework capable of catching fakes from generators that have not yet been invented.

Key Foundational Work

Spoofing detection with DNN and one-class SVM for the ASVspoof 2015 challenge (2015) — contributed 2 concept elements to this direction.
Autoencoder with recurrent neural networks for video forgery detection (2017) — contributed 1 concept elements to this direction.
DeltaPhish: Detecting Phishing Webpages in Compromised Websites (2017) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_one_class_anomaly_detection_anomaly_base_5d12544f,
  title = {one-class anomaly detection (anomaly-based forgery detection)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/one-class-anomaly-detection-anomaly-based-forgery-detection/rdr_5d12544f8929453b9f223f13dcb193a1},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_one_class_anomaly_detection_anomaly_base_5d12544f_v1,
  title = {one-class anomaly detection (anomaly-based forgery detection)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/one-class-anomaly-detection-anomaly-based-forgery-detection/rdr_5d12544f8929453b9f223f13dcb193a1/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake