Anomaly Detection via Likelihood Based Density Estimation

44 concepts42 papers1 proposal

The core challenge of anomaly detection lies in the unpredictable nature of the anomalies themselves. Because unusual events are rare, diverse, and often entirely unseen during training, building a traditional classifier to distinguish "normal" from "abnormal" is incredibly difficult.

Instead of trying to model what is abnormal, a more powerful approach is to model what is normal. By fitting a probabilistic model to represent the normal state of a system, we can estimate its probability density function, $p(x)$. Under this paradigm, any new observation that yields a low likelihood score can be flagged as a statistical outlier. The system does not need to know what a specific failure looks like; it only needs to recognize that the event does not fit the established pattern of normalcy.

The Core Intuition of Normalcy Modeling

By framing anomaly detection as a density estimation task, we shift the problem from drawing a rigid classification boundary to mapping a continuous landscape of probability. High-density regions represent expected system states, while the low-density tails of the distribution represent anomalies.

In Outlier Detection in ECG, Omar Atamny et al. demonstrate this by training Variational Autoencoders (VAEs) exclusively on healthy heart signals. The model maps the normal probability density of cardiac rhythms; when a pathological disturbance occurs, it naturally falls into a low-probability region, generating a high anomaly score.

Similarly, in Green Coverage Detection on Sub-orbital Plantation Images Using Anomaly Detection, Gabriel B. Costa et al. model the statistical normalcy of healthy crops using a multivariate Gaussian distribution. By flagging any pixel falling below a density threshold as an outlier, the authors bypass the need to catalog every possible type of weed or soil variation, demonstrating how a simple probabilistic boundary can handle highly imbalanced real-world environments.

Mapping Latent Spaces with Mixture Models

While raw data can be chaotic, projecting inputs into a structured latent space allows probabilistic models to draw clean density boundaries. A classic and highly effective approach is to pair neural feature extractors with a Gaussian Mixture Model (GMM) to cluster normal behaviors.

For instance, Kieron Messer et al. in Adaptive Automatic Target Recognition with SVM Boosting for Outlier Detection use GMMs to capture the statistical distribution of complex maritime backgrounds, identifying dim moving targets as low-probability outliers against sea glint.

In modern deep learning pipelines, this hybrid approach is frequently applied to deep embeddings:

Even internal network activations can be monitored this way. In Inlier-Centric Post-Training Quantization for Object Detection Models, Minsu Kim et al. use GMMs to separate task-relevant activations from noisy anomalies. Taking this further, Ziv Katzir argues in Detecting Adversarial Perturbations Through Spatial Behavior in Activation Spaces that tracking the Bayesian log-likelihood of class transitions across successive layers can expose adversarial attacks, as malicious inputs trace highly improbable paths through the activation space.

Capturing Temporal and Relational Dynamics

Real-world systems are rarely static; anomalies often manifest as unusual sequences or interactions over time. To handle this, researchers extend density estimation to temporal models like Hidden Markov Models (HMMs), Gaussian Processes (GPs), and Normalizing Flows.

For sequential event logs, Dong Zhang et al. in Semi-Supervised Adapted HMMs for Unusual Event Detection use HMMs to flag video sequences with the lowest likelihood scores under a pre-trained normalcy model. Similarly, Wei Ma et al. in Anomaly based multi-stage attack detection method apply HMMs to event sequences to detect multi-stage cyberattacks when the sequence probability drops below a threshold. For continuous dynamics, Wentong Liao shows in GAUSSIAN PROCESS FOR ACTIVITY MODELING AND ANOMALY DETECTION that modeling traffic regions with Gaussian Processes allows real-time anomaly detection by identifying deviations from a learned predictive distribution.

For highly complex systems with multiple interacting variables, normalizing flows provide a way to compute exact log-densities. In Graph-Augmented Normalizing Flows for Anomaly Detection of Multiple Time Series, Enyan Dai et al. combine normalizing flows with graph structures to isolate anomalies as low-density regions. Furthermore, Dongze Wu et al. in DoFlow: Flow-based Generative Models for Interventional and Counterfactual Forecasting on Time Series utilize continuous normalizing flows to track the exact log-density of future system trajectories, providing early warnings for industrial failures.

Robustness to Noisy Training Data

A major practical advantage of density-based anomaly scoring over classification-based methods is its inherent robustness to imperfect training data. In real-world deployments, it is often impossible to guarantee a perfectly "clean" training set free of anomalies.

Because density estimation focuses on mapping where the bulk of the data lies, sparse, unlabeled anomalies present in the training data naturally fall into low-density regions anyway. As demonstrated by Haoyu Liu et al. in Detecting Multivariate Time Series Anomalies with Zero Known Label, density estimation frameworks remain highly robust even when the training set contains unlabeled outliers.

This property is echoed in Label-Free Multivariate Time Series Anomaly Detection by Qihang Zhou et al., who emphasize that density-based methods avoid the fragile, easily corrupted boundaries of classification models, making them ideal for real-world industrial settings where perfectly clean data is unavailable.

The Likelihood Trap: When Raw Density Fails

The most critical intuition in probabilistic anomaly detection is understanding when raw likelihood fails. A common trap is assuming that any low-probability event is an anomaly, and any high-probability event is normal.

As Oleksandr Shchur et al. observe in Detecting Anomalous Event Sequences with Temporal Point Processes, a high log-likelihood is not a guarantee of normal behavior. In complex distributions, certain highly atypical, repetitive, or simple sequences can paradoxically yield higher probability densities than representative, typical sequences.

To resolve this, models must normalize or contextualize their scores based on the structure of the model itself:

  1. Sequence Length Normalization: In Search Problems in the Domain of Multiplication: Case Study on Anomaly Detection Using Markov Chains, Aviad Cohen explains that raw probabilities in Markov models naturally decay as sequences grow longer, which can cause long, benign sequences to trigger false positives. To fix this, Aviad Cohen proposes normalizing the observed probabilities against the theoretical maximum and minimum path probabilities for that specific sequence length.
  2. Likelihood Ratios: Another strategy is to evaluate relative likelihoods rather than absolute densities. In Learning and Inferring Transportation Routines, Lin Liao et al. track user routines by running a specific routine model in parallel with a generic prior model. Instead of relying on raw density, they monitor the likelihood ratio between the two models. This ensures that deviations (like a missed transit stop) are caught regardless of the route's baseline complexity.

Key Foundational Work

Detecting Errors within a Corpus using Anomaly Detection (2000) — contributed 1 concept elements to this direction.
Adaptive Automatic Target Recognition with SVM Boosting for Outlier Detection (2000) — contributed 1 concept elements to this direction.
Learning and Inferring Transportation Routines (2004) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_likelihood_based_anomaly_scoring_163b78e8,
  title = {likelihood-based anomaly scoring},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/likelihood-based-anomaly-scoring/rdr_163b78e8b06c49289338f6ed5152d3bd},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_likelihood_based_anomaly_scoring_163b78e8_v1,
  title = {likelihood-based anomaly scoring},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/likelihood-based-anomaly-scoring/rdr_163b78e8b06c49289338f6ed5152d3bd/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake