MLAS: Metric Learning on Attributed Sequences

Zhongfang Zhuang, Xiangnan Kong, Elke A. Rundensteiner, Jihane Zouaoui, Aditya Arora

Introduction

Distance metric learning (DML) is a critical component in machine learning, enabling models to quantify the similarity or dissimilarity between data points based on specific feedback. Traditionally, DML has operated in two distinct silos: attribute-based data (fixed-length vectors like user demographics) and sequence-based data (ordered lists like clickstreams).

However, real-world data is often more complex. In web log analysis, a user session is an "attributed sequence" where static context (IP address, browser) and dynamic actions (searches, clicks) are deeply intertwined. Existing methods often fail to capture these cross-domain dependencies—for instance, how a user's device might influence their search behavior. The MLAS (Metric Learning on Attributed Sequences) framework addresses this by explicitly modeling the relationships between what a user is and what a user does.

Methodology

The MLAS framework is built upon three specialized sub-networks designed to process and fuse multi-modal data:

  1. AttNet: A feedforward neural network that learns hierarchical representations from static attribute vectors.
  2. SeqNet: An LSTM (Long Short-Term Memory) network that captures temporal dependencies within sequential data.
  3. FusionNet: The core innovation of the framework, which integrates AttNet and SeqNet. The authors propose three distinct architectural designs for FusionNet to handle different data characteristics:
  4. Balanced Design: Concatenates outputs from both networks into a joint fully connected layer.
  5. AttNet-centric: Feeds sequence representations directly into the attribute network.
  6. SeqNet-centric: Uses attribute representations to condition or initialize the sequence network.
The three proposed FusionNet architectures: Balanced, AttNet-centric, and SeqNet-centric designs.

Finally, a MetricNet layer applies a contrastive loss function to the resulting embeddings. This pulls similar pairs closer in the embedding space while pushing dissimilar pairs apart, effectively learning a distance metric tailored to the attributed sequence data.

Experimental Evaluation

The framework was evaluated on four real-world datasets: AMS-A & AMS-B (travel industry application logs) and Wiki-A & Wiki-B (Wikipedia navigation paths). Performance was measured using Normalized Mutual Information (NMI) to assess clustering quality.

Key Results

MLAS consistently outperformed traditional baselines, including simple concatenation methods (ASF).

  • Performance Gains: MLAS variants achieved an improvement of 18.3% to 25.4% over the best baseline on the AMS datasets and up to 26.3% on the Wikipedia datasets.
  • The Impact of Feedback: A significant finding was the efficiency of the metric learning process. Even with a small amount of expert feedback (e.g., 100 pairs), the model's ability to cluster similar instances improved drastically compared to unsupervised pre-training.
  • Domain-Specific Fusion: The optimal FusionNet architecture varied by domain. The AttNet-centric design was most effective for web logs, while the SeqNet-centric design performed better for Wikipedia navigation, suggesting that the primary signal (context vs. action) differs across applications.
Clustering performance (NMI) across four datasets, comparing MLAS variants against baselines with and without feedback.

Conclusion and Future Work

MLAS demonstrates that modeling the dependencies between attributes and sequences is essential for accurate similarity measures in complex datasets. While the current framework requires empirical testing to select the best fusion architecture, it provides a robust foundation for user profiling and fraud detection. Future research may involve replacing standard LSTMs with Transformer-based architectures and exploring active learning to reduce the reliance on expert-provided feedback pairs.

Summary updated

BibTeX
@inproceedings{zhuang2020mlas,
  author = {Zhongfang Zhuang and Xiangnan Kong and Elke A. Rundensteiner and Jihane Zouaoui and Aditya Arora},
  title = {{MLAS: Metric Learning on Attributed Sequences}},
  booktitle = {IEEE BigData},
  year = {2020},
  doi = {10.1109/bigdata50022.2020.9378490},
}
Figure