MINES: Explainable Anomaly Detection through Web API Invariant Inference

Wenjie Zhang, Yun Lin, Chun Fung Amos Kwok, Xiwen Teoh, Xiaofei Xie, Frank Liauw, Hongyu Zhang, Jin Song DongarXiv 2025 · DOI · arXiv

Introduction: The Gap in Log-Based Detection

Traditional web application security relies heavily on log-based anomaly detection. However, these methods frequently encounter two significant obstacles: Observability Beyond Logs and Log Noise. Standard logs capture API call sequences but often omit the critical state of the underlying database. For instance, a POST /refund request might appear syntactically correct in a log file, yet constitute a malicious attack if the order was never paid or has already been processed for a refund.

Figure 1: Comparison between log-based superficial invariants and database-aware true invariants.

As illustrated in Figure 1, relying solely on log instances can lead to "superficial invariants"—spurious correlations that fail to capture the actual business logic. To address this, a system must look beyond raw logs to the underlying data structures and logical constraints of the application.

The MINES Framework

MINES (Mining Invariants for aNomaly dEtection via Schema) is a novel framework designed to bridge this gap. Instead of learning from noisy log instances, MINES infers "true invariants"—logical business rules—by analyzing API signatures and database schemas using Large Language Models (LLMs).

Figure 5: The MINES workflow pipeline, from schema parsing to invariant generation.

The MINES workflow, depicted in Figure 5, consists of four primary stages:

  1. Schema Parsing: The system extracts API endpoints, parameters, and environmental session data, converting them into a unified "action entity" format. This abstraction allows the system to treat API calls with the same structured rigor as database tables.
  2. Relationship Inference: Leveraging the semantic reasoning capabilities of LLMs (such as GPT-4o), MINES constructs an Augmented Entity-Relationship (ER) Diagram. It identifies semantic links between API parameters and database columns (e.g., mapping a cancelOrder API's orderId to the id primary key in an orders table).
  3. Invariant Generation: The LLM generates candidate invariants as executable Python code. These rules represent fundamental business logic, such as: "A refund is only valid if the corresponding order status in the database is 'paid'."
  4. Invariant Refinement: To mitigate LLM hallucinations or overly restrictive constraints, MINES validates these candidate invariants against a set of known normal logs. Rules that produce false positives are either refined by the LLM or discarded.

At runtime, an offline checker executes these Python-based invariants against incoming logs and database binary logs to detect violations in real-time.

Evaluation and Key Insights

The researchers evaluated MINES across five diverse web applications, including complex microservices like Train-Ticket and platforms written in various languages (Go, Ruby, PHP). A custom web-tamper attack dataset with 31 attack types was used to benchmark performance.

Performance Metrics

MINES demonstrated significant improvements over state-of-the-art baselines like LogRobust and LogFormer:

  • Superior Recall: It achieved 94.8% recall on Train-Ticket and 91.7% on NiceFish, outperforming the best baseline by over 14%.
  • Near-Zero False Positives: The system maintained 100% precision in most scenarios, effectively eliminating the "alert fatigue" common in traditional security tools.
  • High Throughput: Capable of processing 400,000 logs per second, MINES is suitable for high-traffic industrial environments.

The Power of Semantic Reasoning

A key insight from the study is the effectiveness of LLMs in understanding business logic through variable names. Even without explicit documentation, MINES can infer relationships (e.g., matching userId to owner_id) based on semantic similarity. This abstraction to the schema level also reduces input size by up to two orders of magnitude compared to log-based training, making the process more efficient and less prone to noise.

Limitations and Future Directions

While highly effective, MINES has specific dependencies. It relies on semantically meaningful variable names; obfuscated codebases (e.g., func(a, b)) would hinder the LLM's inference capabilities. Additionally, while the framework is generalizable, initial engineering effort is required to adapt it to new database backends or session management styles. Future work may involve deeper integration with existing system documentation to further enhance the detection of complex edge-case attacks.

Conclusion

MINES represents a shift from correlation-based detection to causality-based reasoning. By grounding anomaly detection in the fundamental business logic defined by schemas, it provides a more robust, explainable, and precise solution for securing modern web applications against sophisticated logic-tampering attacks.

Summary updated

Abstract

Detecting the anomalies of web applications, important infrastructures for running modern companies and governments, is crucial for providing reliable web services. Many modern web applications operate on web APIs (e.g., RESTful, SOAP, and WebSockets), their exposure invites intended attacks or unintended illegal visits, causing abnormal system behaviors. However, such anomalies can share very similar logs with normal logs, missing crucial information (which could be in database) for log discrimination. Further, log instances can be also noisy, which can further mislead the state-of-the-art log learning solutions to learn spurious correlation, resulting superficial models and rules for anomaly detection. In this work, we propose MINES which infers explainable API invariants for anomaly detection from the schema level instead of detailed raw log instances, which can (1) significantly discriminate noise in logs to identify precise normalities and (2) detect abnormal behaviors beyond the instrumented logs. Technically, MINES (1) converts API signatures into table schema to enhance the original database shema; and (2) infers the potential database constraints on the enhanced database schema to capture the potential relationships between APIs and database tables. MINES uses LLM for extracting potential relationship based on two given table structures; and use normal log instances to reject and accept LLM-generated invariants. Finally, MINES translates the inferred constraints into invariants to generate Python code for verifying the runtime logs. We extensively evaluate MINES on web-tamper attacks on the benchmarks of TrainTicket, NiceFish, Gitea, Mastodon, and NextCloud against baselines such as LogRobust, LogFormer, and WebNorm. The results show that MINES achieves high recall for the anomalies while introducing almost zero false positives, indicating a new state-of-the-art.
BibTeX
@misc{zhang2025mines,
  author = {Wenjie Zhang and Yun Lin and Chun Fung Amos Kwok and Xiwen Teoh and Xiaofei Xie and Frank Liauw and Hongyu Zhang and Jin Song Dong},
  title = {{MINES: Explainable Anomaly Detection through Web API Invariant Inference}},
  year = {2025},
  eprint = {2512.06906},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.06906},
}
Figure