SEAnet: A Deep Learning Architecture for Data Series Similarity Search

Qitong Wang, Themis Palpanas

Introduction

Similarity search—the process of identifying the most similar data series to a given query—is a fundamental task in time-series analysis, critical to domains such as seismology, astronomy, and neuroscience. As datasets scale to billions of entries, traditional summarization techniques like Symbolic Aggregate approXimation (SAX) often struggle with complex, high-frequency data. SEAnet introduces a novel deep learning framework called Deep Embedding Approximation (DEA) to overcome these limitations. By replacing linear averaging with a specialized autoencoder architecture, SEAnet achieves significantly higher accuracy and efficiency in massive similarity searches.

The current industry standard for data series summarization is SAX, which relies on Piecewise Aggregate Approximation (PAA). PAA simplifies data by taking the mean average of segments. While effective for smooth, highly correlated data, PAA fails on "hard" datasets characterized by high-frequency noise or weak correlations. In these scenarios, PAA produces overly smoothed, indistinguishable summaries, leading to "collisions" in the index and degraded search performance.

Comparison of PAA and SEAnet reconstruction on RandWalk (smooth) vs. Deep1B (complex) series.

As shown in the figure above, while PAA can approximate a random walk series (a), it fails to capture the essential features of a Deep1B series (b), resulting in a nearly flat reconstruction. In contrast, SEAnet's reconstructions remain faithful to the original signal across both types.

Methods: Deep Embedding Approximation (DEA)

SEAnet utilizes a deep neural network architecture designed for DEA. The framework moves beyond simple averaging to learn non-linear representations that preserve the distances between data series.

The DEA-based SAX symbolization workflow, replacing PAA with a learned encoder.

Key Architectural Components

  • Dilated ResNet Backbone: SEAnet employs a full-preactivation ResNet with exponentially increasing dilations. This design provides a massive "receptive field," allowing the model to capture both local nuances and long-range global dependencies without the high computational overhead of standard convolutions.
  • Sum of Squares (SoS) Preservation: A core theoretical contribution of this work is the preservation of total variance. The authors prove that by z-normalizing and appropriately scaling embeddings, the network can be forced to preserve the "Sum of Squares" of the original data. This acts as a powerful regularizer, ensuring that learned embeddings are distinct and informative.
  • SEAtrans and Sampling: For enhanced global context, the authors offer SEAtrans, which integrates Transformer blocks. To handle training on massive scales (up to 100 million series), they developed SEAsam and SEAsamE, sampling techniques that focus on representative and "hard" pairs to optimize the learning process.

Key Insights and Results

The effectiveness of SEAnet stems from its ability to act as a "non-linear PCA," focusing on dimensions that contain the most information. The authors found that the decoder, while not used during the search itself, is a vital regularizer during training that prevents embedding collapse.

Performance Metrics

The authors evaluated SEAnet against massive real-world and synthetic datasets, including Seismic, Astro, and Deep1B.

  • Search Accuracy: SEAnet improved the "1st BSF (Best-So-Far) tightness"—a measure of how close the first approximate result is to the true nearest neighbor—in 61 out of 63 experimental settings.
  • Index Quality: On the challenging Deep1B dataset, SEAnet's index was 14% more compact than traditional SAX, indicating more effective grouping of similar items.
  • Efficiency: SEAnet converges 2.9x to 3.35x faster than other deep architectures like InceptionTime, making it viable for large-scale industrial applications.
  • Versatility: Beyond search, DEA embeddings outperformed PAA in k-NN classification tasks on approximately 68% of the UCR Time Series Classification Archive datasets.

Conclusion and Future Work

SEAnet represents a significant leap forward in data series summarization. By leveraging deep learning to preserve variance and capture complex patterns, it provides a robust alternative to traditional linear methods. While currently optimized for approximate similarity search, future work aims to establish "lower bounding" properties for exact search and explore incremental learning for dynamic datasets.

Summary updated

Abstract

A key operation for massive data series collection analysis is similarity search. According to recent studies, SAX-based indexes offer state-of-the-art performance for similarity search tasks. However, their performance lags under high-frequency, weakly correlated, excessively noisy, or other dataset-specific properties. In this work, we propose Deep Embedding Approximation (DEA), a novel family of data series summarization techniques based on deep neural networks. Moreover, we describe SEAnet, a novel architecture especially designed for learning DEA, that introduces the Sum of Squares preservation property into the deep network design. We further enhance SEAnet with SEAtrans encoder. Finally, we propose novel sampling strategies, SEAsam and SEAsamE, that allow SEAnet to effectively train on massive datasets. Comprehensive experiments on 7 diverse synthetic and real datasets verify the advantages of DEA learned using SEAnet in providing high-quality data series summarizations and similarity search results.
BibTeX
@article{wang2023seanet,
  author = {Qitong Wang and Themis Palpanas},
  title = {{SEAnet: A Deep Learning Architecture for Data Series Similarity Search}},
  journal = {IEEE Trans. Knowl. Data Eng.},
  year = {2023},
}
Figure