Abstract
This paper introduces FRODA, a framework for Open-set Domain Adaptation (OSDA) that identifies and ignores unknown classes in target data by factorizing representations into shared and private subspaces. By utilizing group sparsity to force target samples into either a "known" or "unknown" category, the method achieves state-of-the-art accuracy while operating an order of magnitude faster than previous approaches.
Introduction
Most Domain Adaptation (DA) techniques operate under a "closed-set" assumption: they presume the source (training) and target (test) data share the exact same set of categories. In real-world scenarios, this assumption is frequently violated. For instance, a robot deployed in a novel environment will encounter "unknown" objects never seen during training. Standard models often erroneously classify these new objects into known categories with high confidence.
Open-set Domain Adaptation (OSDA) addresses this by aligning the distributions of known classes across domains while simultaneously detecting and discarding samples from "unknown" classes. This paper proposes FRODA to solve this challenge efficiently through subspace factorization.
Methodology
The core intuition behind FRODA (Factorized Representations for Open-set Domain Adaptation) is that data can be decomposed into two distinct subspaces:
- Shared Subspace ($V$): Captures features common to the known classes in both the source and target domains.
- Private Subspace ($U$): Captures features unique to the "unknown" classes found only in the target domain.
Technical Framework
The target data is modeled as $X_t \approx [V, U]T$, where $T$ contains the coefficients. The "secret sauce" of the method is Group Sparsity. The authors apply a penalty that encourages each target sample to be reconstructed by either the shared subspace or the private subspace, but not a mixture of both. This creates a clear mathematical separation between known and unknown samples.
An extended version, D-FRODA (Discriminative FRODA), incorporates a linear classifier term. This ensures that the shared representation is not only effective for reconstruction but also highly discriminative for distinguishing between known classes. The model is optimized using an alternating minimization strategy, solving for subspaces, coefficients, and classifier weights iteratively.
Experimental Results
The authors evaluated FRODA on two major benchmarks: the BCIS Dataset (Bing, Caltech256, ImageNet, SUN) and the classic Office Dataset (Amazon, DSLR, Webcam).
Performance and Efficiency
On the BCIS dataset, D-FRODA outperformed the previous state-of-the-art method, ATI, by significant margins. In specific tasks like SUN $\to$ Bing, the accuracy improvement exceeded 32 percentage points.
Beyond accuracy, FRODA offers a substantial computational advantage. While the competing ATI method required approximately 32 seconds to converge, FRODA finished in roughly 2.5 seconds—a 10x speedup. This efficiency stems from treating OSDA as a subspace learning problem rather than relying on expensive iterative sample assignments.
Parameter Robustness
A key practical advantage of FRODA is its relative insensitivity to the threshold ($\epsilon$) used to separate known from unknown samples. As long as the threshold is above a minimal value, the mean accuracy remains stable, simplifying deployment in varied environments.
Conclusion and Future Work
FRODA demonstrates that factorized representations and group sparsity provide a powerful, efficient mechanism for open-set domain adaptation. While currently relying on linear subspaces, the framework is highly effective for deep features like DeCAF. Future work may explore non-linear manifold learning or kernelized versions to handle more complex data distributions, as well as integrating non-linear classifiers directly into the optimization loop.
Summary updated