Cross-Domain Knowledge Transfer Using Structured Representations

Samarth Swarup, Sylvian R. RayAAAI 2006

Abstract

Traditional transfer learning often struggles when source and target tasks have different input or output dimensions. This paper presents a neuroevolutionary framework for cross-domain knowledge transfer that focuses on structural logic rather than specific feature weights. By evolving network topologies and employing graph mining to identify recurring functional motifs, the system builds a library of "knowledge primitives." These primitives can be reused across tasks with varying dimensions, significantly accelerating learning and improving final performance on complex Boolean logic problems.

Introduction: The Challenge of Cross-Domain Transfer

A fundamental goal in artificial intelligence is the ability to transfer knowledge between disparate domains. While humans can easily draw analogies—for instance, using the flow of water in pipes to understand electrical current—most machine learning models are constrained by fixed input and output spaces. In standard neural networks, a model trained on a 4-input task cannot be directly applied to a 16-input task because the underlying weight matrices are incompatible.

This research addresses the problem of cross-domain transfer: enabling an agent to learn the underlying structural logic of a problem so it can solve similar challenges in environments with entirely different dimensions.

Methodology: Structured Neuroevolution and Graph Mining

The proposed "cumulative learning" framework avoids the rigid constraints of fixed-topology networks by focusing on the evolution of network structure.

1. Structured Neuroevolution

Instead of optimizing weights via backpropagation, the system evolves the topology of sparse, many-layered neural networks. Individual nodes have fixed weights, and learning occurs by discovering the optimal wiring of these nodes into a directed acyclic graph (DAG).

Figure 1: Representation of a node and its integration into a network topology.

2. Identifying Knowledge Primitives

Once an agent successfully solves a task, a graph mining algorithm (CloseGraph) is used to identify frequent sub-networks, or "motifs," that appear across multiple successful solutions. These motifs represent functional logic—such as detecting adjacency—that is independent of the total number of inputs.

3. Cumulative Learning

Discovered motifs are added to a library of "knowledge primitives." In subsequent tasks, the evolutionary algorithm can utilize these high-level building blocks as single units during mutation. This allows the agent to construct complex solutions from pre-validated logic rather than starting from individual nodes.

Experimental Setup: Boolean Adjacency Tasks

The framework was evaluated using a series of Boolean function tasks where the goal is to output 1 if the input vector contains k adjacent 1s. To test cross-domain transfer, tasks were spread across four input dimensions (4, 8, 12, and 16) and varying values of k (2 to 5). Ten tasks were learned sequentially, forcing the agent to transfer structural logic from smaller input spaces to significantly larger ones.

Key Results and Discussion

The inclusion of transferred knowledge primitives provided substantial benefits across all tested domains.

Faster Learning and Increased Robustness

Agents equipped with the primitive library reached lower error rates significantly faster than those starting from scratch. Furthermore, the transfer-enabled runs exhibited lower variance, suggesting that the primitives provide a reliable "head start" that stabilizes the evolutionary process.

Superior Generalization

The most striking result was the reduction in "true error" (testing error)—the error rate when the learned classifier is tested on all possible input combinations.

Figure 4: Comparison of testing error between agents with and without knowledge transfer across various task configurations.

As shown in Figure 4, transfer-enabled agents consistently outperformed the "no-transfer" baselines. The performance gap was particularly pronounced in higher-dimensional tasks (e.g., 12-input and 16-input domains), where the search space for a solution from scratch is prohibitively large.

Conclusion and Future Work

This research demonstrates that focusing on network topology rather than specific feature weights allows for effective knowledge transfer across domains with different dimensions. By building a library of structural primitives, agents can achieve a "cumulative advantage," making complex tasks easier to solve as more knowledge is acquired.

Future work will focus on refining complexity control to prevent overfitting as the primitive library grows. Additionally, the authors aim to apply this framework to more complex domains like robotics, where primitives could represent fundamental physical movements that can be composed into sophisticated behaviors.

Summary updated

BibTeX
@inproceedings{swarup2006cross,
  author = {Samarth Swarup and Sylvian R. Ray},
  title = {{Cross-Domain Knowledge Transfer Using Structured Representations}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2006},
}
Figure