Imagine taking a 100-question exam with a peculiar grading rule: the exam is divided into a few long sections, and if you get just one question right within a section, you receive a 100% score for that entire section. If you don't know any of the answers, your best strategy isn't to study harder; it is to guess randomly as many times as possible. The more shots you take, the higher your chances of hitting at least one correct answer per section and acing the exam.
This is the exact flaw underlying the Point Adjustment (PA) protocol, a dominant evaluation metric in time-series anomaly detection. As exposed by Siwon Kim et al. in Towards a Rigorous Evaluation of Time-Series Anomaly Detection, this protocol creates a severe performance illusion where random noise or untrained models can achieve near-perfect F1-scores, masking a lack of actual algorithmic progress in the field.
The Mechanics of Metric Inflation
In time-series data, anomalies rarely occur as isolated, single-timestamp spikes. Instead, they usually manifest as continuous ranges, segments, or volumes (Paul Boniol et al., New Trends in Time Series Anomaly Detection).
Under standard point-wise evaluation, if an anomalous event lasts for 100 timestamps and a model flags only a single timestamp within that window, its recall for that segment is a meager 1%. Because real-world labels are often imprecise or slightly shifted, researchers introduced the Point Adjustment heuristic to avoid overly penalizing models for minor temporal misalignments. The rule is simple: if a model correctly identifies any single point within an anomalous segment, the entire segment is retroactively marked as a success.
Ground Truth Anomaly: [========================================] (100 points)
Model Prediction: [ . ] (1 point detected)
Point-Adjusted Label: [========================================] (100 points credited)
While well-intentioned, this protocol introduces a massive mathematical bias. As the length of an anomalous segment increases, the probability that a random prediction or a noisy signal happens to intersect with that segment approaches 100%. Once a single intersection occurs, the model is credited with detecting the full duration of the event.
Consequently, metrics like the point-adjusted F1-score fail to evaluate whether a model has captured the actual structure, duration, or boundaries of an anomaly (Ayan Banerjee et al., Detection of unknown unknowns in autonomous systems). Instead, it heavily favors highly volatile, noisy models that output frequent positive predictions (Qinghua Liu et al., The Elephant in the Room: Towards A Reliable Time-Series Anomaly Detection Benchmark).
The Illusion of Progress
The widespread adoption of point adjustment has severely distorted the perceived state of art. Multiple systematic evaluations have revealed that many complex deep learning architectures owe their high performance scores entirely to this metric artifact.
- Random Predictors Outperforming Deep Learning: Dennis Wagner et al. demonstrated in TimeSeAD: Benchmarking Deep Multivariate Time-Series Anomaly Detection that stochastic noise generators can achieve top-tier F1-scores under PA evaluation, outperforming sophisticated neural networks.
- Superiority of Simple Baselines: When evaluating models without the artificial boost of point adjustment, the "progress" of recent years often vanishes. M. Saquib Sarfraz et al. in Position: Quo Vadis, Unsupervised Time Series Anomaly Detection? showed that complex models frequently fail to beat simple linear baselines like PCA once PA is removed. Similarly, Astha Garg et al. found that a basic univariate autoencoder paired with a robust scoring function can outperform state-of-the-art multivariate models (An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series).
- Masking Real-World Failures: In practical deployments, knowing exactly when an anomaly starts and ends is critical. By marking an entire window as detected based on a single point, PA obscures poor localization and high false-positive rates (Nesryne Mejri et al., Unsupervised anomaly detection in time-series: An extensive evaluation and analysis of state-of-the-art methods). A model could trigger a single alarm at a random timestamp and be credited for a week-long industrial anomaly, despite completely missing its onset and duration.
Designing Unbiased Metrics
To establish a rigorous foundation for time-series anomaly detection, researchers are shifting away from standard point adjustment toward metrics that respect temporal context without rewarding random guessing.
The $PA\%K$ Protocol
To mitigate the all-or-nothing leniency of traditional PA, Siwon Kim et al. proposed the $PA\%K$ protocol. Instead of requiring just one correct point, a model must correctly identify at least $K\%$ of the timestamps within an anomalous segment before receiving segment-level credit. This forces the model to capture a meaningful portion of the anomaly's duration, preventing sparse noise spikes from inflating the score.
Affiliation Metrics
Rather than relying on arbitrary segment-level adjustments, Alexis Huet et al. introduced Affiliation Metrics in Local Evaluation of Time Series Anomaly Detection Algorithms. This approach calculates mathematical "zones of affiliation" based on the distance between predicted and ground-truth events. It provides a localized, parameter-free evaluation of precision and recall that cannot be gamed by fragmenting predictions or exploiting long anomaly windows.
Volume Under the Surface (VUS)
Another threshold-independent alternative gaining traction is the Volume Under the Surface (VUS) metric, highlighted by Paul Boniol et al. as a robust replacement for traditional F1-scores. By evaluating performance across varying temporal window sizes and detection thresholds simultaneously, VUS prevents metric inflation and provides a more reliable measure of a model's true discriminative capabilities (Beibu Li et al., CrossAD: Time Series Anomaly Detection with Cross-scale Associations and Cross-window Modeling).
By adopting these rigorous evaluation frameworks, the field is moving away from metric-driven illusions and toward architectures that genuinely understand the complex temporal dynamics of real-world systems.
Key Foundational Work
Precision and Recall for Time Series (2018) — contributed 2 concept elements to this direction.