Abstract
RINS-T is a novel, unsupervised deep learning framework designed to reconstruct corrupted time series data—such as those with missing values or heavy outliers—without requiring any pre-training on external datasets. By combining the "Deep Prior" effect of neural architectures with robust optimization techniques like the Huber loss, it significantly outperforms traditional signal processing and existing deep learning baselines in real-world, "messy" data scenarios.
Introduction
Time series data in fields like finance, healthcare, and engineering are rarely "clean." They are frequently plagued by sensor failures (outliers), transmission errors (missing values), and environmental interference (noise). Reconstructing the original signal $x$ from a corrupted observation $y$ is known as an inverse problem.
Standard approaches often fall short in these scenarios. Traditional statistical methods like Least Squares (LS) assume Gaussian noise and collapse when faced with heavy-tailed noise or outliers. Supervised deep learning models require massive amounts of labeled training data and often fail under distribution shifts. Existing unsupervised methods, such as 1D-DIP, use neural network structure as a prior but still typically rely on Gaussian assumptions and struggle with stability.
Methods
RINS-T (Robust Implicit Neural Solvers for Time Series) introduces a robust framework that treats the neural network architecture itself as the only prior knowledge needed. The core innovations include:
- Huber Loss Integration: Instead of standard Mean Squared Error, RINS-T uses the Huber loss. This function is quadratic for small residuals (handling Gaussian noise) but becomes linear for large residuals, making the model much less sensitive to outliers.
- Guided Input Initialization: Most "Deep Prior" models start with random noise. RINS-T instead starts with a smoothed version of the corrupted signal, helping the network focus on refining details rather than learning basic signal structure from scratch.
- Input Perturbation: During optimization, the model adds small amounts of noise to its input. This acts as a regularizer, preventing the network from overfitting to specific noise artifacts.
- Convex Output Combination: To prevent jittery convergence, the model's output at each step is a weighted blend of the current and previous iterations.
The underlying architecture is a hierarchical 1D CNN (U-Net style) with skip connections, which possesses an inherent "inductive bias" toward structured patterns over random noise.
Evaluation and Results
The authors evaluated RINS-T across four diverse datasets: Audio (dolphin vocalizations), Electricity consumption, Solar power production, and EEG brain activity. RINS-T consistently outperformed baselines like Wavelet denoising, Total Variation (TV), and 1D-DIP across several challenging scenarios.
In mild noise scenarios, RINS-T achieved the highest Signal-to-Noise Ratio (SNR). Its true strength was demonstrated in the presence of 10% outliers, where it maintained high accuracy while traditional filters and standard 1D-DIP failed. Furthermore, the "Guided Input" allowed RINS-T to converge faster than standard Deep Prior methods, increasing its practical utility.
Discussion and Conclusion
A key insight of this work is the exploitation of the "spectral bias" of CNNs: when fitting both a structured signal and random noise, the network fits the signal much faster. RINS-T leverages this to extract the signal before noise is memorized. The authors also provide a probabilistic justification, proving that using Huber loss is mathematically equivalent to a Maximum A Posteriori (MAP) estimate for a mixture of Gaussian and Laplace noise.
While RINS-T is robust, it can sometimes oversmooth very sharp transients in complex signals. Future work will explore incorporating wavelet-inspired layers to capture high-frequency details and extending the framework to non-linear inverse problems.
Summary updated