TL;DR
Researchers at NEC Corporation have introduced a new framework for Partial Time Series Classification (pTSC), where data is often fragmented or observed over inconsistent intervals. By introducing two novel modules—Adaptive Multi-scale Pooling (AMSpool) and Temporal Encoding (TE)—they allow a single CNN model to handle variable-length inputs and varying timestamps without the accuracy-killing effects of heavy padding or feature collapse.
What Problem Does This Paper Address?
In real-world scenarios, like tracking particles in a moving liquid or monitoring sensors, we rarely get a "perfect" time series. Observations are often interrupted, leading to fragments of different lengths starting at different times. The authors call this partial Time Series Classification (pTSC).
Standard Convolutional Neural Networks (CNNs) struggle here because of the Receptive Field (RF) trade-off. If a CNN is deep (large RF), it can capture long-term patterns but requires heavy "padding" for short sequences, which contaminates the data. If it is shallow (small RF), it handles short sequences well but fails to see the "big picture" in long ones. Furthermore, most models ignore when a fragment was recorded, losing vital context about the overall process.
Methods
The authors propose two modules that can be integrated into existing 1D-CNN architectures:
- Adaptive Multi-scale Pooling (AMSpool): Instead of forcing every input through every layer, AMSpool adaptively selects features from different depths. For short sequences, it uses features from early layers (small RF). For long sequences, it aggregates features from deeper layers (large RF). It uses a Recurrent Module (LSTM) to aggregate these multi-scale features into a fixed-length vector for final classification.
- Temporal Encoding (TE): Inspired by positional encoding in Transformers, TE embeds the absolute timestamp of each observation directly into the input. By concatenating time-dependent vectors to the raw data, the model learns to distinguish a fragment recorded at the start of a process from one recorded at the end.
To ensure the model only learns from "real" data, they implement masking in the Global Average Pooling (GAP) and Batch Normalization layers, effectively ignoring the padded regions during calculation.
Datasets
The authors evaluated their approach on two main fronts:
- Private Trajectory Dataset: A real-world dataset involving the detection of foreign matter (anomalies) vs. air bubbles (normal) in liquid vaccines. Trajectories are often fragmented due to occlusions or tracking failures.
- UCR/UEA Time Series Archive: A standard public benchmark. They selected a subset of univariate and multivariate datasets (e.g., Crop, ElectricDevices, InsectWingbeat) and simulated pTSC by randomly cropping the sequences.
Key Insights and Ideas
- The "RF-Length Match" is Critical: The most significant "aha" moment is that a fixed architecture is inherently biased toward a specific input length. By truncating the feature extraction process before the Receptive Field exceeds the input length, AMSpool prevents "feature collapse" caused by excessive padding.
- Time is a Feature, Not Just an Axis: In many processes (like the rotating liquid containers in their study), the dynamics change over time. TE allows the model to realize that a specific movement pattern might mean "anomaly" at $t=100$ but "normal bubble" at $t=500$.
- Multi-scale is Better than Global: Rather than just using the final layer's output, aggregating features from multiple intermediate blocks (multi-scale) provides a richer representation that is more robust to length variations.
Key Results
- Accuracy Boost on Short Data: On the trajectory dataset, the proposed AMSCNN-te model improved balanced accuracy on short sequences from 84.8% (baseline) to 89.5%.
- Statistical Significance: Using the Steel-Dwass test, the authors confirmed that the improvements on short and middle-length data were statistically significant ($p < 0.001$).
- Robustness: On the UCR/UEA benchmarks, the model showed a higher number of "wins" against standard ResNet baselines, particularly on datasets where the input length was comparable to or shorter than the model's default receptive field.
Limitations and Future Work
While the modules significantly improve performance on fragmented data, there are a few trade-offs:
- Computational Overhead: The masking processes in GAP and Batch Normalization, along with the additional LSTM layer in AMSpool, increase training and inference time (e.g., inference jumped from 2.2ms to 9.5ms in some tests).
- Long Sequence Performance: In some cases, the proposed model performed slightly worse than a standard fixed-length CNN on very long sequences. The authors suggest that refining how the model handles long-term temporal correlations is a path for future research.
- Masking Complexity: The current implementation requires careful tracking of "valid" regions through the network, which adds complexity to the code compared to standard global pooling.
Summary updated