AnomalyGen: Enhancing Log-Based Anomaly Detection with Code-Guided Data Augmentation

Xinyu Li, Yintong Huo, Chenxi Mao, Shiwen Shan, Yuxin Su, Yanlin Wang, Zibin ZhengarXiv 2026 · arXiv

TL;DR

AnomalyGen solves the severe data sparsity problem in log-based anomaly detection by automatically synthesizing high-quality, labeled training logs directly from source code. By combining log-oriented static analysis with LLM Chain-of-Thought (CoT) reasoning, it uncovers rare but valid execution paths, boosting anomaly detection F1-scores by up to 15.2% and drastically reducing false alarms.

What Problem Does This Paper Address?

Log-based anomaly detection is crucial for maintaining the reliability of modern distributed systems, but it faces a major bottleneck: training data sparsity. The authors conducted an empirical study revealing that popular public benchmark datasets cover less than 10% of the log templates actually defined in system source code (for instance, the widely used HDFS dataset covers a microscopic 0.99%).

Because deep learning models only learn from what they see, they frequently misclassify unseen but perfectly valid execution paths as anomalies in production, triggering costly false alarms. Traditional data augmentation methods like random perturbation (shuffling logs) or resampling fail because they only rearrange existing data without expanding template coverage. Meanwhile, pure static analysis methods generate "hollow" template stubs that lack realistic runtime parameters (like block IDs or IP addresses) needed by modern semantic-based detection models.

Methods

The authors propose AnomalyGen, a white-box framework that bridges the gap between static code structure and dynamic runtime reality using a three-phase pipeline:

  1. Phase I: Log-Oriented Static Analysis: The framework prunes the system's global call graph to isolate methods that actually emit logs. It then performs an Abstract Syntax Tree (AST) and dominance analysis to construct Log-Oriented Control Flow Graphs (LCFGs), which map out the structurally valid orderings and conditional constraints of logs within each method.
  2. Phase II: Cross-Method Sequence Assembly with CoT Verification: AnomalyGen uses a virtual stack-based simulation to chain logs across different methods. Because structural reachability doesn't guarantee logical feasibility, the framework feeds the code context into a Large Language Model (LLM) using Chain-of-Thought (CoT) prompting. The LLM verifies if the execution path is logically consistent and, if valid, generates realistic runtime parameters to fill the log templates.
  3. Phase III: Knowledge-Driven Labeling and Augmentation: The generated sequences are automatically labeled as normal or anomalous using domain-knowledge heuristics (e.g., checking for ERROR/FATAL severity levels or keywords like "timeout"). These high-quality synthetic sequences are then carefully blended into the training set.
Fig. 1. The three-stage workflow of AnomalyGen. Phase I performs log-based graph pruning (1), subgraph extraction (2), and LCFG construction (3) to capture intra-procedural logic. Phase II reconstructs inter-procedural paths via stack-based simulation (4) and merges log sequences with CoT semantic verification (5). Phase III completes the pipeline with automatic anomaly annotation (6) and controlled data augmentation (7) to produce high-quality labeled sessions.

Datasets

The framework was evaluated using two prominent public benchmarks from the LogHub suite that have open-source Java codebases:

  • HDFS: Over 11 million log messages collected from an Amazon EC2 cluster, segmented by block ID.
  • Zookeeper: Approximately 74,380 log messages from a 32-node lab cluster, covering synchronization behaviors like leader election.

By using AnomalyGen, the authors scaled the HDFS training set's log event coverage from 30 templates to 2,874 templates—achieving 99.48% coverage of the source code's total log templates.

Key Insights and Ideas

  • Data Coverage is the Real Bottleneck: No matter how expressive or advanced a deep learning architecture is, it cannot reliably distinguish between a true anomaly and a valid execution path it has simply never seen before.
  • Structure + Logic = High Fidelity: Static analysis defines the boundaries of what structurally can happen, while LLMs provide the logical verification of what makes sense at runtime. As shown in the ablation studies, removing either component severely degrades data quality.
  • Sequence-Aware Models Benefit Most: Models that encode the temporal ordering of logs (like Transformers and LSTMs evaluating next_log predictions) benefit significantly more from AnomalyGen than models using purely semantic text encodings. This proves that AnomalyGen successfully captures real program execution flows.

Key Results

The authors evaluated AnomalyGen across 12 different anomaly detection models.

  • Massive Performance Gains: Unsupervised deep learning models saw the highest jumps. Notably, an unsupervised Transformer model on HDFS improved its F1-score from 0.818 to 0.970 (an absolute gain of 15.2 percentage points).
  • Crucial Components: Ablation experiments proved the necessity of the framework's core phases. Omitting the static analysis phase (w/o analysis) dropped the Transformer's F1-score by 8.7 points. Omitting the LLM reasoning phase (w/o cot) caused a 10.7 point drop, because the LLM without CoT generated logically impossible paths that introduced harmful noise into training.
Fig. 5. Example of Phase II: a caller context and candidate callee path are provided to the LLM, which evaluates logical consistency and generates parameter values.
  • High Efficiency: AnomalyGen is highly practical for real-world deployment. The one-time static analysis setup takes under 15 minutes, and the on-demand sequence generation process is highly parallelizable across multi-core servers.

Limitations and Future Work

  • LLM Stochasticity: Despite setting the temperature to 0, LLMs are inherently stochastic and carry a residual risk of hallucinating unreachable paths or generating structurally flawed sequences.
  • Heuristic Labeling Limits: The rule-based labeling in Phase III relies on explicit keywords and severity levels, meaning it might miss subtle, application-specific zero-day faults that don't trigger standard error keywords.
  • White-Box Constraints: The tool currently requires full access to source code, limiting its use on closed-source third-party libraries (though the authors note bytecode analysis tools like Soot could bypass this in the future).
  • Future Directions: The current implementation is Java-specific. Future work will focus on expanding AnomalyGen to other languages (C++, Python, Go) and integrating symbolic execution to handle complex, data-dependent paths that challenge LLM reasoning alone.

Summary updated

Abstract

Log-based anomaly detection is fundamentally constrained by training data sparsity. Our empirical study reveals that public benchmark datasets cover less than 10% of source code log templates. Consequently, models frequently misclassify unseen but valid execution paths as anomalies, leading to false alarms. To address this, we propose AnomalyGen, a novel framework that augments training data by synthesizing labeled log sequences from source code. AnomalyGen combines log-oriented static analysis with Large Language Model (LLM) reasoning in three stages: (1) building Log-Oriented Control Flow Graphs (LCFGs) to enumerate structurally valid execution paths; (2) applying LLM Chain-of-Thought (CoT) reasoning to verify logical consistency and generate realistic runtime parameters (e.g., block IDs, IP addresses); and (3) labeling generated sequences with domain heuristics. Evaluations on HDFS and Zookeeper across 12 diverse anomaly detection models show AnomalyGen consistently improves performance. Deep learning models achieved average F1-score gains of 2.18% (HDFS) and 1.69% (Zookeeper), with an unsupervised Transformer on HDFS jumping from 0.818 to 0.970. Ablation results show that both static analysis and LLM-based verification are necessary: removing them reduces F1 by up to 8.7 and 10.7 percentage points, respectively. Our framework and datasets are publicly available to facilitate future research.
BibTeX
@misc{li2026anomalygen,
  author = {Xinyu Li and Yintong Huo and Chenxi Mao and Shiwen Shan and Yuxin Su and Yanlin Wang and Zibin Zheng},
  title = {{AnomalyGen: Enhancing Log-Based Anomaly Detection with Code-Guided Data Augmentation}},
  year = {2026},
  eprint = {2604.11107},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2604.11107},
}

Loading related content...

Figure