Learning Multi-Way Relations via Tensor Decomposition With Neural Networks

Koji Maruhashi, Masaru Todoriki, Takuya Ohwa, Keisuke Goto, Yu Hasegawa, Hiroya Inakoshi, Hirokazu AnaiAAAI 2018 · DOI

Abstract

DeepTensor is a novel framework designed for the classification of multi-way relational data, such as network logs and financial transactions. By integrating a supervised tensor decomposition method, Structure Restricted Tensor Decomposition (SRTD), with neural networks, DeepTensor extracts index-independent structural features. This approach allows for high accuracy in complex, high-order data environments while maintaining interpretability through contribution scores. Experimental results across intrusion detection, P2P lending, and bioinformatics demonstrate that DeepTensor significantly outperforms traditional tensor and graph-based classification methods.

Introduction

Many real-world datasets are inherently "multi-way," involving relationships between multiple types of entities. For example, a network log links a source IP, a destination IP, and a port. While this data can be represented as a tensor (a multi-dimensional array), standard classification methods struggle when specific entities change. In cybersecurity, an attacker might rotate IP addresses to avoid detection; traditional tensor classification assumes that a specific index always represents the same entity. DeepTensor addresses this by extracting index-independent features—patterns of connectivity that signal an attack regardless of the specific IPs or ports involved.

Methodology

DeepTensor introduces two core innovations to bridge the gap between tensor decomposition and supervised learning:

  1. Structure Restricted Tensor Decomposition (SRTD): Standard Tucker decomposition breaks a tensor into a "core tensor" and "factor matrices" in an unsupervised manner. SRTD, however, forces the decomposition to align with a target core tensor ($V$). This target tensor acts as an abstract template for important relational structures, such as "one-to-many" scanning patterns.
  2. Extended Backpropagation (EBP): To optimize the system, the authors developed EBP, which allows the model to update the target core tensor $V$ using classification errors. This ensures the decomposition process is explicitly tuned to distinguish between different classes (e.g., attack vs. normal traffic).
The DeepTensor architecture, illustrating the integration of SRTD and Extended Backpropagation for supervised learning.

By using factor matrices to map specific entities to the core tensor, the model identifies structural patterns even when the entities involved are entirely new to the system.

Experimental Evaluation

The model was evaluated across three distinct domains:

  • Intrusion Detection (IDS): Using the DARPA 1998 dataset to identify DOS and Probing attacks via 4th-order tensors (sIP, dIP, sPort, dPort).
  • P2P Lending (LEND): Predicting high-risk transactions based on relations between lenders, borrowers, and lending IDs.
  • Bioinformatics (BIO): Predicting biological activity in chemical compounds and proteins represented as graphs/tensors.

Key Results

DeepTensor demonstrated superior accuracy across all tasks. In the IDS "Probing" task, it achieved an average precision of 78.52%, significantly outperforming standard Tucker-based neural networks (62.10%) and graph-based methods (40.13%). In the P2P lending domain, DeepTensor's precision was nearly double that of the next best neural network approach.

Interpretability and Insights

A significant advantage of DeepTensor is its interpretability. By calculating "Contribution Scores" (CS), the model can highlight exactly which entities in a log were responsible for a specific classification. This provides security engineers with a clear roadmap for investigating potential threats.

Contribution scores identifying specific malicious IP addresses and ports within network traffic logs.

The authors found that the advantage of DeepTensor over traditional graph kernels grows as the "order" of the data increases. While graphs are effective for pairwise relations, tensors are superior for capturing the complex, multi-entity interactions found in modern datasets.

Conclusion

DeepTensor provides a robust solution for classifying high-order relational data by focusing on the "shape" of interactions rather than specific entity identities. While future work is needed to automate hyperparameter tuning and explore different local surrogate models for explanation, DeepTensor represents a major step forward in making complex tensor-based models both accurate and interpretable.

Summary updated

BibTeX
@inproceedings{maruhashi2018learning,
  author = {Koji Maruhashi and Masaru Todoriki and Takuya Ohwa and Keisuke Goto and Yu Hasegawa and Hiroya Inakoshi and Hirokazu Anai},
  title = {{Learning Multi-Way Relations via Tensor Decomposition With Neural Networks}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2018},
  doi = {10.1609/aaai.v32i1.11760},
}
Figure