Abstract
Domain Adaptation (DA) is a critical technique for transferring knowledge from a labeled source domain to an unlabeled target domain. However, standard DA methods often struggle when the target data is corrupted by severe, unpredictable noise, such as contiguous occlusions or high-density pixel corruption. This paper introduces Robust Information-Theoretic Domain Adaptation (RIDA), a framework that leverages correntropy to measure reconstruction error. By utilizing the localized similarity properties of correntropy, RIDA automatically identifies and de-emphasizes noisy samples during the learning process. Experimental results demonstrate that RIDA maintains high accuracy even under 70% pixel corruption, significantly outperforming traditional distance-based adaptation methods.
Introduction
The fundamental goal of Domain Adaptation is to find a shared feature space where source and target distributions align. While effective for clean datasets, most existing algorithms assume that noise in the target domain is either negligible or follows a predictable sparse pattern. In real-world applications—such as facial recognition under varying shadows or object detection with partial occlusions—noise is often dense and structured. When these algorithms attempt to align corrupted target data with clean source data, they inadvertently "align the noise," leading to a breakdown in feature representation.
The RIDA Framework
To address the limitations of standard distance metrics like the Frobenius norm, RIDA introduces three core technical innovations:
- Correntropy-based Reconstruction: Unlike global metrics that are sensitive to outliers, correntropy is a localized similarity measure. It assigns high weight to samples that can be reconstructed accurately and effectively ignores samples with large, unpredictable errors (the noise).
- Relative Entropy Regularization: To ensure the learned transformation is non-trivial and scale-invariant, the model incorporates a regularization term based on relative entropy.
- Half-Quadratic (HQ) Optimization: Because information-theoretic objectives are non-convex, the authors employ an HQ optimization technique. This approach decomposes the complex objective into iterative quadratic sub-problems, with a mathematical guarantee of convergence.
Experimental Setup and Datasets
The resilience of RIDA was tested against severe noise conditions using two primary benchmarks:
- COIL-20 (Object Recognition): Modified with contiguous 16x16 white block occlusions and random pixel corruption ranging up to 70%.
- CMU-PIE (Face Recognition): Modified with natural illumination changes and "unrelated image occlusions" (e.g., replacing parts of a face with a monkey image) to simulate complex real-world interference.
Key Results and Insights
Resilience to Extreme Noise
One of the most striking findings is RIDA's performance on the COIL-20 dataset. At 70% random pixel corruption, where traditional transfer methods typically fail, RIDA maintained an accuracy of over 75%. This demonstrates that the information-theoretic approach is far more robust than methods relying on $l_1$-norm sparsity assumptions.
Adaptive Sample Weighting
The algorithm's success stems from its ability to naturally "mute" outliers. During the update phase, RIDA assigns low weights to contaminated samples without requiring a predefined noise model. This adaptive weighting allows the model to focus exclusively on the clean components of the target domain.
Performance and Efficiency
In face recognition tasks (CMU-PIE), RIDA achieved a 10% improvement in accuracy over established competitors like Joint Distribution Adaptation (JDA). Furthermore, the HQ optimization scheme proved computationally efficient, typically reaching a stable local minimum in fewer than 10 iterations.
Conclusion and Future Work
RIDA provides a theoretically sound and practically robust solution for domain adaptation under severe noise. While the current model uses a linear transformation matrix, future research could extend this framework to non-linear or deep neural network architectures. Additionally, while the model is robust, the sensitivity to hyperparameters ($\lambda_1, \lambda_2, \sigma$) suggests that automated parameter tuning could further enhance its applicability to diverse, real-world datasets.
Summary updated