Introduction
Predicting daily human behavior or power consumption is inherently challenging because real-life schedules are rarely perfectly aligned. Temporal distortions—such as a commuter catching a later bus or a household shifting its peak energy usage—create misalignments in time-series data that traditional statistical models and motif-discovery algorithms struggle to resolve. This paper introduces TeNet (Temporal embedding enhanced convolutional neural Network), a novel architecture designed to automatically realign distorted time-series data to its dominant underlying patterns, enabling robust regression from partial daily sequences.
The TeNet Architecture
The core innovation of TeNet is its four-layer design, which prioritizes the discovery of abstract "snippets" or recurring behavioral routines while remaining resilient to temporal shifts.
- Temporal Embedding Layer: Unlike standard convolutional layers that share weights across a sequence, this layer utilizes individualized weights for each time step. It encodes each data point alongside its immediate temporal neighbors, effectively acting as a supervised denoising step that "shifts" distorted data back into alignment.
- 1D Convolution and Max-Pooling: This stage functions as a suite of snippet detectors. It identifies local, recurring abstract patterns of moderate length within the realigned time-series.
- Sigmoid Layer: This layer serves as a non-linear feature extractor, reducing dimensionality while preserving salient information.
- L1-Regularized Least-Squares: The final regression layer employs L1 regularization to enforce sparsity. This ensures the model bases its predictions on the most distinct, high-level features rather than noisy signals.
Experimental Setup and Datasets
The model was evaluated across 15 individual datasets representing three distinct modalities:
- Human Mobility (HMD/HMT): Daily traveling distance and time extracted from GPS data.
- Household Power Consumption (HPC): High-resolution smart meter data from households in France and Australia.
- Synthetic Data: A dataset created by artificially distorting Australian power data to specifically test the temporal embedding layer's realignment capabilities.
Performance Analysis
TeNet was benchmarked against several competitive baselines, including Support Vector Regression (with Linear, RBF, and Polynomial kernels) and Multiple Kernel Regression (MKR). The primary task was to predict total daily accumulation using only data available up to 2:00 PM.
TeNet demonstrated clear dominance, achieving the best performance on nearly all datasets. On average, it provided a 13% increase in HitRate@20% and a 9.1% decrease in Mean Relative Error (MRE) compared to the next-best performing method. An ablation study on synthetic data further highlighted the importance of the temporal embedding layer, which reduced Mean Squared Error (MSE) from 15.5 to 6.4 when added to a standard CNN.
Key Insights and Discussion
- Automated Realignment: Empirical evidence suggests the temporal embedding layer learns to reverse schedule shifts. For instance, if a routine is delayed, the layer learns weights that effectively push the data back to its standard temporal slot.
- Sparse Feature Learning: Visualization of the CNN filters indicates that the network learns sparse, distinct "snippets"—abstract patterns with clear spikes and valleys—rather than dense, noisy representations.
- Complexity Scaling: The researchers observed a strong positive correlation between TeNet's performance advantage and the entropy (complexity) of the dataset. As data becomes messier and sample sizes grow, TeNet's ability to extract abstract features allows it to scale more effectively than traditional models.
Conclusion
TeNet introduces a robust framework for time-series regression by combining individualized temporal embedding with the feature-extraction power of CNNs. By learning to "undo" temporal distortions, the model captures the underlying routines of human behavior and power consumption more accurately than traditional regression techniques. Future work may extend this architecture to general $k$-ahead forecasting and broader time-series prediction tasks.
Summary updated