Sparse Regularization for Cross Domain Transfer

44 concepts29 papers

When transferring knowledge across different domains, the fundamental challenge is separating the signal from the noise. If a model attempts to transfer everything—every feature, every source dataset, or every parameter update—it inevitably suffers from negative transfer, where irrelevant source information degrades target performance.

To combat this, a powerful paradigm has emerged: sparse regularization. By enforcing sparsity (via $L_1$, Elastic Net, or mixed norms), models are forced to find the minimal, most critical pathways for cross-domain transfer. Sparsity acts as an implicit selection mechanism across three distinct granularities: mapping heterogeneous features, selecting relevant source domains, and constraining domain transformations.

Aligning Heterogeneous Feature Spaces via Sparse Mappings

When source and target domains have completely different feature representations (such as text versus images, or different sensor configurations), we must learn a transformation matrix to map them into a shared space. A dense transformation matrix assumes every target feature is a combination of all source features, which quickly leads to overfitting, especially when target labels are scarce.

Enforcing sparsity changes this assumption. As shown by Shaobo Han et al. in their work on Cross-Domain Multitask Learning with Latent Probit Models, hierarchical sparsity priors on transformation matrices reduce the upper bound of classifier estimation error by isolating minimal, relevant mappings. This aligns with physical and semantic realities: a target feature typically corresponds to only a small, localized subset of source features. For instance, Sanatan Sukhija et al. demonstrate in Supervised Heterogeneous Domain Adaptation via Random Forests that a specific sensor in one environment maps naturally to a few related sensors in another, rather than a dense blend of the entire array.

To formalize this mathematically, Joey Tianyi Zhou et al. frame the discovery of cross-domain feature transformations as a compressed sensing problem in Heterogeneous Domain Adaptation for Multiple Classes, where individual classifier weights serve as sparse measurements of the underlying mapping. Similarly, Jie Tang et al. incorporate Laplace-driven sparsity within a shared latent space in Cross-Domain Ranking via Latent Space Learning to filter out domain-specific noise and isolate transferable ranking correlations.

For unsupervised scenarios where target labels are entirely missing, Yuhong Guo leverages mixed-norm regularizers (such as $l_{1,1}$ or $l_{1,2}$ norms) in Unsupervised Heterogeneous Domain Adaptation with Sparse Feature Transformation to ensure each target feature is reconstructed from a highly constrained subset of source features, preventing spurious correlations when parallel data is limited. Furthermore, Yong Luo et al. combine sparsity with non-negativity constraints in Heterogeneous Multitask Metric Learning Across Multiple Domains and Heterogeneous Multi-task Metric Learning across Multiple Domains to narrow the search space and improve the interpretability of these cross-domain mappings.

Mitigating Negative Transfer through Sparse Source Selection

Often, the challenge isn't just mapping features, but deciding which source domains or examples to trust. In multi-source domain adaptation, aggregating all available data blindly introduces massive noise. Sparsity acts as a hard filter to zero out irrelevant sources.

For example, when adapting web images to video recognition, Lixin Duan et al. introduce a "Domain Selection Machine" in Exploiting web images for event recognition in consumer videos: A multiple source domain adaptation approach. Their data-dependent sparse regularizer selects only a subset of source classifiers consistent with the target distribution. Similarly, in camera on-boarding for person re-identification, Sk Miraj Ahmed et al. apply sparsity constraints to source model weights in Camera On-Boarding for Person Re-Identification Using Hypothesis Transfer Learning, effectively zeroing out contributions from dissimilar camera environments.

This selection mechanism is equally vital when borrowing individual examples or dealing with massive source libraries:

This selective transfer philosophy is also mirrored in deep architectures and collaborative filtering. Guangneng Hu et al. apply $L_1$ regularization to cross-domain transfer layers in CoNet: Collaborative Cross Networks for Cross-Domain Recommendation to filter out cross-domain noise, while Jian Cheng et al. use group sparsity in Recommendation by Mining Multiple User Behaviors with Group Sparsity to explicitly separate shared transferable latent factors from private, behavior-specific ones.

Preventing Over-Transportation in Domain Transformation

A third facet of this research direction focuses on domain regeneration or sample transformation—modifying data or parameters so that the target distribution matches the source distribution. Without regularization, transformation models suffer from "over-transportation," changing parts of the data that should remain invariant (such as altering the background of an image during a style transfer).

Farzan Farnia et al. address this directly in Sparse Domain Transfer via Elastic Net Regularization. By enforcing sparsity on the transformation function, the model performs implicit feature selection, modifying only the salient variables required to transition between domains (e.g., specific keywords in sentiment reversal) while preserving the underlying content structure.

This conservative update strategy is crucial for fine-grained tasks like facial micro-expression recognition across databases. Xiaohua Huang 0003 et al. introduce a Target Sample Re-Generator (TSRG) in Learning a Target Sample Re-Generator for Cross-Database Micro-Expression Recognition, which is further expanded by Yuan Zong et al. in Domain Regeneration for Cross-Database Micro-Expression Recognition. By using $l_1$-norm regularization on the feature reconstruction matrix, the authors force the model to rebuild target samples from a sparse set of relevant source features, preventing the model from simply blurring data to match distributions. The authors expand on this in Cross-Database Micro-Expression Recognition: A Benchmark via Region Selective Transfer Regression (RSTR), combining distribution alignment with sparsity to isolate a minimal subset of critical facial regions.

Without Sparsity (Dense Mapping):
Target Feature ───► Blended/Noisy Combination of ALL Source Features (Overfitting)

With Sparsity (L1 / Mixed Norms):
Target Feature ───► [Selected Feature 1] ───► Direct Semantic Alignment
               ───► [Irrelevant Feature] ───► Zeroed Out (Noise Filtered)

Finally, this principle extends to temporal shifts and causal inference. To handle concept drift in streaming data, André Artelt et al. utilize $L_1$ regularization in Unsupervised Unlearning of Concept Drift with Autoencoders to ensure that only the specific features driving the distributional shift are modified. In causal effect estimation, Ronald Moore et al. regularize the difference between target and source parameters using an $l_1$ norm in Transfer Causal Learning: Causal Effect Estimation with Knowledge Transfer, assuming that the underlying mechanisms are largely shared and forcing the model to focus strictly on a sparse set of divergent causal features.

Key Foundational Work

Exploiting web images for event recognition in consumer videos: A multiple source domain adaptation approach (2012) — contributed 2 concept elements to this direction.
Cross-Domain Multitask Learning with Latent Probit Models (2012) — contributed 2 concept elements to this direction.
Transfer Learning by Borrowing Examples for Multiclass Object Detection (2011) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_sparse_source_selection_612ce8c5,
  title = {sparse source selection},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/sparse-source-selection/rdr_612ce8c567fe4f0bac743432f0b0e005},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_sparse_source_selection_612ce8c5_v1,
  title = {sparse source selection},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/sparse-source-selection/rdr_612ce8c567fe4f0bac743432f0b0e005/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}

See something wrong? Report a mistake