Mitigating Negative Transfer in Cross Domain Learning

1,679 concepts1,113 papers2 proposals

The core promise of cross-domain learning is intuitive: if a model lacks data in a target domain, it can borrow knowledge from a richer, related source domain. For instance, an e-commerce platform trying to recommend items to a "cold-start" user in a new market might look at that same user's browsing history in an established market.

However, this assumption frequently breaks down in practice. When the behavioral patterns, cultural contexts, or underlying distributions of the source and target domains diverge, transferring knowledge indiscriminately can actively harm performance. This phenomenon is known as negative transfer or cross-domain interference. Instead of helping, the source domain introduces noise, leading to worse predictive accuracy than if the model had been trained on the sparse target data alone.

To build models that safely learn across multiple domains, researchers must design mechanisms that understand what to share, when to share it, and how to suppress harmful interference.

The Root Cause: Feature Entanglement and Parameter Pollution

Why does negative transfer happen? In standard multi-domain or transfer learning setups, models often share a single latent space or a rigid set of parameters across all tasks. When heterogeneous data is pooled together without boundaries, the unique quirks of different domains collide.

As Qiang Sheng et al. observe in Memory-Guided Multi-View Multi-Domain Fake News Detection, forcing a single model architecture to share rigid parameters across highly diverse domains causes conflicting signals to confuse the model. For example, the stylistic patterns that indicate fake news in a political domain might be completely normal in an entertainment domain.

Jiajie Zhu et al. formalize this as "information entanglement" in Domain Disentanglement with Interpolative Data Augmentation for Dual-Target Cross-Domain Recommendation. When a model fails to separate preferences that are genuinely shared across domains from those that are strictly domain-specific, the specific quirks of one domain pollute the recommendations of another.

This interference becomes even more pronounced when there is a data imbalance. As Xiaopeng Li et al. point out in HAMUR: Hyper Adapter for Multi-Domain Recommendation, a dominant, data-rich domain can easily overwhelm the nuances of smaller, sparser domains during joint training, leading to severe performance degradation in the smaller domains.

Indiscriminate Pooling:
[Source Domain Data] \
                      ---> [Shared Latent Space] ---> Conflicting Signals (Negative Transfer)
[Target Domain Data] /

Disentangled Architecture:
[Source Domain Data] ---> [Domain-Specific Filter] \
                                                    ---> [Shared Latent Space] (Clean Transfer)
[Target Domain Data] ---> [Domain-Specific Filter] /

Strategy 1: Filtering Noise via Information Bottlenecks

One of the most effective ways to prevent negative transfer is to restrict the flow of information, forcing the model to compress representations so that only universally applicable features pass through.

Jiangxia Cao leverages this intuition in Cross-Domain Recommendation to Cold-Start Users via Variational Information Bottleneck. By applying the Information Bottleneck (IB) principle, the model is penalized for retaining excess information from the source domain. The bottleneck acts as a filter: it strips away the platform-specific quirks (such as a user's specific UI interactions on a source site) while retaining only the core, abstract user preferences that genuinely map to the target domain.

Strategy 2: Latent Space Edge Pruning and Attention Gates

Instead of compressing the entire representation, another approach is to actively identify and discard harmful data points or relationships in the latent space.

In graph-based cross-domain recommendation, user-item interactions are mapped as edges. However, not all source edges are relevant. In Refining Graph Representation for Cross-Domain Recommendation Based on Edge Pruning in Latent Space, Taisei Hirakawa et al. use anomaly detection within a shared latent space to identify "noisy" source edges—interactions that do not align with target domain behaviors—and intentionally prune them.

Similarly, selective attention mechanisms can be deployed to dynamically weight cross-domain signals. Chenglin Li et al. demonstrate in One for All, All for One: Learning and Transferring User Embeddings for Cross-Domain Recommendation that combining a domain-invariant global profile with a selective attention mechanism allows the system to choose which cross-domain features to listen to and which to ignore, protecting the target domain from irrelevant noise.

Strategy 3: Explicit Task Relationship Modeling and Adapters

Rather than assuming all domains can share the same parameters, advanced architectures isolate domain-specific knowledge using modular components, such as adapters, or explicitly calculate how tasks relate to one another.

In Transfer metric learning by learning task relationships, Yu Zhang et al. introduced a framework that models task relationships using a covariance matrix. By embedding this matrix directly into the optimization problem, the model mathematically determines whether a source task is positively correlated, irrelevant, or negatively correlated with the target task, automatically suppressing counterproductive biases.

For deep learning architectures, pluggable adapters offer a structural solution to parameter pollution. The HAMUR framework by Xiaopeng Li et al. utilizes a shared hyper-network to dynamically generate instance-specific weights for domain-specific adapters. This architecture allows the model to capture shared knowledge across diverse domains while maintaining strict structural boundaries, ensuring that unique domain characteristics are preserved without mutual interference.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_negative_transfer_cross_domain_interfere_328c949c,
  title = {negative transfer (cross-domain interference)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/negative-transfer-cross-domain-interference/rdr_328c949c891849f88ec1bd596ea4a576},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_negative_transfer_cross_domain_interfere_328c949c_v1,
  title = {negative transfer (cross-domain interference)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/negative-transfer-cross-domain-interference/rdr_328c949c891849f88ec1bd596ea4a576/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake