Convolutional Neural Networks for Time-dependent Classification of Variable-length Time Series

Azusa Sawada, Taiki Miyagawa, Akinori F. Ebihara, Shoji Yachida, Toshinori Hosoi

TL;DR

Researchers at NEC Corporation have introduced a new framework for Partial Time Series Classification (pTSC), where data is often fragmented or observed over inconsistent intervals. By introducing two novel modules—Adaptive Multi-scale Pooling (AMSpool) and Temporal Encoding (TE)—they allow a single CNN model to handle variable-length inputs and varying timestamps without the accuracy-killing effects of heavy padding or feature collapse.

What Problem Does This Paper Address?

In real-world scenarios, like tracking particles in a moving liquid or monitoring sensors, we rarely get a "perfect" time series. Observations are often interrupted, leading to fragments of different lengths starting at different times. The authors call this partial Time Series Classification (pTSC).

Standard Convolutional Neural Networks (CNNs) struggle here because of the Receptive Field (RF) trade-off. If a CNN is deep (large RF), it can capture long-term patterns but requires heavy "padding" for short sequences, which contaminates the data. If it is shallow (small RF), it handles short sequences well but fails to see the "big picture" in long ones. Furthermore, most models ignore when a fragment was recorded, losing vital context about the overall process.

Methods

The authors propose two modules that can be integrated into existing 1D-CNN architectures:

  1. Adaptive Multi-scale Pooling (AMSpool): Instead of forcing every input through every layer, AMSpool adaptively selects features from different depths. For short sequences, it uses features from early layers (small RF). For long sequences, it aggregates features from deeper layers (large RF). It uses a Recurrent Module (LSTM) to aggregate these multi-scale features into a fixed-length vector for final classification.
  2. Temporal Encoding (TE): Inspired by positional encoding in Transformers, TE embeds the absolute timestamp of each observation directly into the input. By concatenating time-dependent vectors to the raw data, the model learns to distinguish a fragment recorded at the start of a process from one recorded at the end.

To ensure the model only learns from "real" data, they implement masking in the Global Average Pooling (GAP) and Batch Normalization layers, effectively ignoring the padded regions during calculation.

Datasets

The authors evaluated their approach on two main fronts:

  • Private Trajectory Dataset: A real-world dataset involving the detection of foreign matter (anomalies) vs. air bubbles (normal) in liquid vaccines. Trajectories are often fragmented due to occlusions or tracking failures.
  • UCR/UEA Time Series Archive: A standard public benchmark. They selected a subset of univariate and multivariate datasets (e.g., Crop, ElectricDevices, InsectWingbeat) and simulated pTSC by randomly cropping the sequences.

Key Insights and Ideas

  • The "RF-Length Match" is Critical: The most significant "aha" moment is that a fixed architecture is inherently biased toward a specific input length. By truncating the feature extraction process before the Receptive Field exceeds the input length, AMSpool prevents "feature collapse" caused by excessive padding.
  • Time is a Feature, Not Just an Axis: In many processes (like the rotating liquid containers in their study), the dynamics change over time. TE allows the model to realize that a specific movement pattern might mean "anomaly" at $t=100$ but "normal bubble" at $t=500$.
  • Multi-scale is Better than Global: Rather than just using the final layer's output, aggregating features from multiple intermediate blocks (multi-scale) provides a richer representation that is more robust to length variations.

Key Results

  • Accuracy Boost on Short Data: On the trajectory dataset, the proposed AMSCNN-te model improved balanced accuracy on short sequences from 84.8% (baseline) to 89.5%.
  • Statistical Significance: Using the Steel-Dwass test, the authors confirmed that the improvements on short and middle-length data were statistically significant ($p < 0.001$).
  • Robustness: On the UCR/UEA benchmarks, the model showed a higher number of "wins" against standard ResNet baselines, particularly on datasets where the input length was comparable to or shorter than the model's default receptive field.

Limitations and Future Work

While the modules significantly improve performance on fragmented data, there are a few trade-offs:

  • Computational Overhead: The masking processes in GAP and Batch Normalization, along with the additional LSTM layer in AMSpool, increase training and inference time (e.g., inference jumped from 2.2ms to 9.5ms in some tests).
  • Long Sequence Performance: In some cases, the proposed model performed slightly worse than a standard fixed-length CNN on very long sequences. The authors suggest that refining how the model handles long-term temporal correlations is a path for future research.
  • Masking Complexity: The current implementation requires careful tracking of "valid" regions through the network, which adds complexity to the code compared to standard global pooling.

Summary updated

Abstract

Time series data are often obtained only within a limited time range due to interruptions during observation process. To classify such partial time series, we need to account for 1) the variable-length data drawn from 2) different timestamps. To address the first problem, existing convolutional neural networks use global pooling after convolutional layers to cancel the length differences. This architecture suffers from the trade-off between incorporating entire temporal correlations in long data and avoiding feature collapse for short data. To resolve this trade-off, we propose Adaptive Multi-scale Pooling, which aggregates features from an adaptive number of layers, i.e., only the first few layers for short data and more layers for long data. Furthermore, to address the second problem, we introduce Temporal Encoding, which embeds the observation timestamps into the intermediate features. Experiments on our private dataset and the UCR/UEA time series archive show that our modules improve classification accuracy especially on short data obtained as partial time series.
BibTeX
@inproceedings{sawada2022convolutional,
  author = {Azusa Sawada and Taiki Miyagawa and Akinori F. Ebihara and Shoji Yachida and Toshinori Hosoi},
  title = {{Convolutional Neural Networks for Time-dependent Classification of Variable-length Time Series}},
  booktitle = {IJCNN},
  year = {2022},
  url = {https://openreview.net/forum?id=JPBVGNKKJNf},
}

Loading related content...

Figure