Introduction
Unsupervised Domain Adaptation (UDA) in deep learning aims to transfer knowledge from a labeled "source" domain to an unlabeled "target" domain. For time series data, such as wearable sensor readings or ECGs, UDA faces two unique hurdles: temporal dynamics and channel-wise shifts. Traditional pseudo-labeling strategies, often borrowed from computer vision, typically ignore sequential dependencies and treat domain shifts as a "black box." This can lead to noisy pseudo-labels, especially when multi-sensor setups experience hardware differences or placement shifts.
To address these limitations, researchers have introduced TransPL (Transition-based Pseudo-Labeling), a novel UDA framework that replaces black-box pseudo-labeling with an explainable approach based on temporal transitions. By modeling time series patches as discrete codes and tracking their transitions via Markov chains, TransPL explicitly accounts for temporal dynamics and channel-wise sensor shifts.
Methodology: The TransPL Framework
TransPL operates through a three-stage pipeline designed to capture and transfer temporal transition patterns across domains.
1. Coarse and Fine Vector Quantization (VQ)
Instead of processing raw continuous signals, TransPL breaks time series into patches and maps them to discrete "codes" using a VQ-VAE-inspired architecture. This mapping is hierarchical:
- Coarse Codebook: Captures essential, low-frequency "motifs" or global trends of the signal.
- Fine Codebook: Captures high-frequency residuals and fine-grained details.
2. Modeling Transitions via Markov Chains
The core innovation of TransPL is treating the sequence of coarse codes as a Markov chain. The model builds Transition Matrices (TMs) that represent the probability of one state (code) transitioning to another:
- Class-wise TMs: Learned from the source domain to define the characteristic temporal transitions of each class.
- Channel-wise TMs: Constructed for both source and target domains to identify how specific sensors (channels) shift across domains.
3. Explainable Pseudo-Labeling via Bayes' Rule
To label target data, TransPL employs Bayes' rule, calculating the likelihood of a target sequence belonging to a specific class based on the source's transition patterns. To handle sensor shifts, the framework uses Optimal Transport to compare source and target channel-wise TMs. If a channel's transition patterns have shifted significantly, the model assigns it a lower "Channel Alignment Score," prioritizing reliable sensors for the final prediction.
Key Insights and Results
TransPL was evaluated on four major time series benchmarks: UCIHAR, WISDM, and HHAR (human activity recognition), as well as PTB (electrocardiogram classification).
Performance Gains
TransPL consistently outperformed state-of-the-art baselines like SHOT and CoDATS. It achieved an average 6.1% improvement in pseudo-label accuracy and a 4.9% improvement in Macro-F1 scores across the benchmarks. Furthermore, the framework naturally accommodates weak supervision (prior knowledge of the target domain's label distribution), which widens its performance gap over competitors.
Coarse Codes for Stable Transitions
By analyzing Permutation Entropy (PE), the authors demonstrated that coarse codes represent simpler, more stable global trends. This makes them significantly more effective for modeling robust temporal transitions than high-entropy fine codes.
Selective Channel Adaptation
Not all sensors are equally affected by domain shifts. By weighting channels based on their transition alignment, TransPL avoids being misled by a single noisy or shifted sensor, such as a loosely fitted smartwatch gyroscope.
Explainability
Unlike standard neural networks, TransPL provides interpretable insights into the domain adaptation process. Researchers can inspect the transition matrices to understand why a model classified a segment a certain way.
Limitations and Future Work
While highly effective, TransPL has two primary areas for future improvement:
- Information vs. Shift Trade-off: Currently, TransPL down-weights channels that shift heavily. However, a heavily shifted channel might still contain critical classification information. Future iterations could integrate "channel importance" metrics to balance this trade-off.
- Mathematical Regularization: The coarse/fine hierarchy works well empirically but lacks a strict mathematical constraint to guarantee that coarse codes always capture the most general patterns. Adding specific regularization terms could make this hierarchical representation more robust.
Summary updated