Securing Your Transactions: Detecting Anomalous Patterns In XML Documents

Eitan Menahem, Alon Schclar, Lior Rokach, Yuval EloviciarXiv 2012 · arXiv

Abstract

XML serves as a critical medium for data exchange in modern information systems, yet its complexity often leaves it vulnerable to semantic and structural anomalies. This paper introduces XML-AD, a specialized framework designed to detect these anomalies by combining automated feature extraction with a novel "multi-univariate" learning algorithm called ADIFA. By decomposing high-dimensional XML structures into individual attribute density functions, XML-AD achieves a high detection rate (89%) with a minimal false positive rate (0.2%), effectively overcoming the "curse of dimensionality" that plagues traditional multivariate methods.

The Challenge of XML Anomaly Detection

While standard security protocols like encryption and digital signatures protect data in transit, they do not safeguard against semantic attacks—such as SQL injection or schema poisoning—that occur before these measures are applied. Detecting these anomalies is technically difficult due to the high dimensionality of XML documents, which can contain thousands of unique elements. In such high-dimensional spaces, traditional distance-based anomaly detection fails because the distance between normal and anomalous points converges, making them indistinguishable.

The XML-AD Framework

The proposed XML-AD framework addresses these challenges through a structured three-stage pipeline.

The XML-AD framework architecture

1. Feature Extraction

The process begins with the Transactions Features Extractor. It parses the XML Schema Definition (XSD) to understand the expected structure and data types (e.g., numerical, string, enumeration). It then extracts features from a corpus of XML transactions, creating a feature matrix that captures the raw content of the documents.

2. Lossy Flattening and Dataset Generation

Because XML documents vary in size and complexity, they must be transformed into a uniform format suitable for machine learning. The Dataset Generator employs "lossy flattening," using aggregate functions—such as Max, Min, Sum, and TF-IDF for text—to condense complex data into fixed-length vectors. This approach preserves critical information while significantly reducing dimensionality.

The structure of the flattened dataset

3. The ADIFA Algorithm

The core of the system is the Attribute DensIty Function Approximation (ADIFA) algorithm. Unlike traditional multivariate approaches, ADIFA learns a separate univariate density function for every single attribute. It then aggregates these individual "normality scores" using either a geometric or arithmetic mean. This "multi-univariate" strategy allows the system to pinpoint exactly which attribute caused an anomaly, providing valuable localization for security audits.

Experimental Evaluation

The framework was tested across diverse datasets, including real-world insurance and inventory transactions, as well as network traffic (ARP packets) converted to XML. To simulate threats, an Anomaly Injection module introduced attacks like value poisoning, XSS, and XPath injection.

Key Results

  • Performance: XML-AD achieved a 89% True Positive Rate with a remarkably low 0.2% False Positive Rate.
  • Scalability: ADIFA consistently outperformed established algorithms like One-Class SVM and Global Density Estimation (GDE), which struggled with high-dimensional data.
  • Efficiency: The algorithm proved fast enough for real-time transaction monitoring, outperforming other multi-univariate adaptations.

Conclusion and Future Directions

XML-AD demonstrates that a "multi-univariate" approach is superior for high-dimensional XML data. The use of lossy flattening proves sufficient for detecting structural and content-based attacks without requiring a perfect representation of the original document. Future work aims to evolve this framework into a real-time "XML-Firewall" and explore association-rule learning to detect sophisticated attacks involving uncommon combinations of otherwise normal values.

Summary updated

Abstract

XML transactions are used in many information systems to store data and interact with other systems. Abnormal transactions, the result of either an on-going cyber attack or the actions of a benign user, can potentially harm the interacting systems and therefore they are regarded as a threat. In this paper we address the problem of anomaly detection and localization in XML transactions using machine learning techniques. We present a new XML anomaly detection framework, XML-AD. Within this framework, an automatic method for extracting features from XML transactions was developed as well as a practical method for transforming XML features into vectors of fixed dimensionality. With these two methods in place, the XML-AD framework makes it possible to utilize general learning algorithms for anomaly detection. Central to the functioning of the framework is a novel multi-univariate anomaly detection algorithm, ADIFA. The framework was evaluated on four XML transactions datasets, captured from real information systems, in which it achieved over 89% true positive detection rate with less than a 0.2% false positive rate.
BibTeX
@misc{menahem2012securing,
  author = {Eitan Menahem and Alon Schclar and Lior Rokach and Yuval Elovici},
  title = {{Securing Your Transactions: Detecting Anomalous Patterns In XML Documents}},
  year = {2012},
  eprint = {1209.1797},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1209.1797},
}
Figure