Towards Adapting ImageNet to Reality: Scalable Domain Adaptation with Implicit Low-rank Transformations

Erik Rodner, Judy Hoffman, Jeff Donahue, Trevor Darrell, Kate SaenkoarXiv 2013 · arXiv

Abstract

Computer vision models trained on curated internet datasets often struggle when applied to real-world scenes due to "domain shift." This paper introduces a scalable optimization method for domain adaptation that learns a transformation between different image distributions. By leveraging an implicit low-rank structure, the proposed method achieves linear-time complexity, enabling it to scale to massive datasets and transfer classifiers to new categories without target-domain labels.

Introduction: The Challenge of Domain Shift

Most computer vision models are trained on "object-centric" datasets like ImageNet, where objects are high-resolution, centered, and often against clean backgrounds. However, when these models are deployed in "scene-centric" environments, such as the SUN2012 database, performance significantly degrades. Real-world images are frequently blurred, occluded, or truncated.

Comparison between scene-centric (SUN 2012) and object-centric (ImageNet) domains.

This phenomenon is known as domain shift. While domain adaptation techniques exist, they typically face two major hurdles:

  1. Scalability: Many methods are computationally expensive, making them impractical for large-scale data.
  2. Category Limitation: Traditional methods often require labeled examples for every category in both domains.

Methodology: Scalable Max-Margin Domain Transform

The authors build upon the Max-Margin Domain Transform (MMDT) framework, which learns a linear transformation matrix $W$ to map target-domain features into the source-domain space. The core innovation lies in a novel optimization strategy that ensures efficiency:

  • Dual Coordinate Descent: The transformation learning is reformulated as a vectorized SVM-like problem, allowing the use of efficient coordinate descent solvers.
  • Implicit Low-Rank Structure: The authors prove that the optimal transformation matrix $W$ has a rank at most equal to the number of object categories. By representing $W$ implicitly, the computational cost of updates is reduced from $O(D \cdot \tilde{D})$ to $O(\tilde{D})$, where $D$ and $\tilde{D}$ are feature dimensionalities.

This approach allows for category-invariant adaptation. Since the transformation is learned across multiple categories, it captures the general "style" of the domain shift, allowing it to be applied to entirely new categories not seen during the adaptation phase.

Experimental Evaluation

The method was evaluated on two primary benchmarks:

  • ImageNet ILSVRC2010 to SUN2012: A challenging task involving 84 matched categories to test adaptation from web images to real-world scene bounding boxes.
  • Bing to Caltech-256: A standard benchmark where the source domain consists of web-crawled images.

Key Results and Performance

The proposed LS-MMDT method demonstrates significant improvements in both speed and accuracy.

Recognition rate and learning time comparison.
  • Efficiency: As shown in the results, the new solver achieves state-of-the-art accuracy while being orders of magnitude faster than the original MMDT (seconds versus minutes or hours).
  • Zero-Shot Transfer: In scenarios with new categories, the method successfully adapted models even with zero target training labels. This "Transferred" model often outperformed "Oracle" SVMs trained on up to 100 target-domain labels.
  • Robustness: On the ImageNet-to-SUN2012 task, the method provided a consistent performance boost over standard SVMs, particularly when target data was scarce.

Conclusion and Future Directions

While highly effective, the current approach focuses on linear transformations. Future work may explore non-linear mappings to handle more complex domain shifts, though maintaining scalability remains a challenge. Additionally, while the study utilized Bag-of-Words features, the underlying optimization logic is potentially applicable to the final layers of modern deep neural networks for efficient fine-tuning.

Summary updated

Abstract

Images seen during test time are often not from the same distribution as images used for learning. This problem, known as domain shift, occurs when training classifiers from object-centric internet image databases and trying to apply them directly to scene understanding tasks. The consequence is often severe performance degradation and is one of the major barriers for the application of classifiers in real-world systems. In this paper, we show how to learn transform-based domain adaptation classifiers in a scalable manner. The key idea is to exploit an implicit rank constraint, originated from a max-margin domain adaptation formulation, to make optimization tractable. Experiments show that the transformation between domains can be very efficiently learned from data and easily applied to new categories. This begins to bridge the gap between large-scale internet image collections and object images captured in everyday life environments.
BibTeX
@misc{rodner2013towards,
  author = {Erik Rodner and Judy Hoffman and Jeff Donahue and Trevor Darrell and Kate Saenko},
  title = {{Towards Adapting ImageNet to Reality: Scalable Domain Adaptation with Implicit Low-rank Transformations}},
  year = {2013},
  eprint = {1308.4200},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1308.4200},
}
Figure