Abstract
Traditional machine learning algorithms often suffer from a static inductive bias, limiting their ability to generalize across diverse tasks. This paper introduces the Trans-Dimensional Learner (TDL), a meta-learning system that automatically adapts its learning bias by recruiting and reusing previously learned features. Experiments on a complex 29-function logic suite demonstrate that learning a diverse set of problems can be more efficient than learning a narrow subset, revealing a "knowledge scaffolding" effect.
Introduction
In machine learning, the "learning bias" or inductive bias represents the set of assumptions an algorithm uses to generalize from data. A static bias often creates a bottleneck: an algorithm optimized for linear tasks may fail on complex parity problems. To achieve more general intelligence, systems must move beyond "one-shot learning"—where the model is reset after every task—and instead adopt meta-learning strategies that allow the adaptation of bias over time based on cumulative experience.
The Trans-Dimensional Learner (TDL)
The authors propose the Trans-Dimensional Learner (TDL), an evolution of the Evolutionary Growth Perceptron (EGP). TDL is characterized by its ability to dynamically construct its own architecture. Rather than relying on a fixed number of layers or neurons defined by a human designer, TDL recruits new hidden units in a bottom-up manner as needed.
A core innovation of TDL is Feature Reuse. When faced with a new task, the system evaluates its existing "pool" of previously learned features. It uses a quality measure ($Q_{k,i}$) to rank these features, balancing their accuracy on the current task against the "cost" of their depth in the network hierarchy. By connecting to relevant "old" features, TDL effectively shifts its learning bias to suit the new problem's structure.
Experimental Results: The Power of Multi-Task Learning
The system was evaluated using the "29-fkt Problem," a suite of 29 functions ranging from simple logic gates (AND, OR) to complex 7-bit parity and binary adders. A "Parity Subset" served as a control group to test the benefits of multi-task learning.
The results revealed a counter-intuitive phenomenon: the network learned the entire 29-function set faster than it learned the parity functions alone. Specifically, the parity-only task required approximately 270 generations, whereas the full 29-function set was solved in under 210 generations.
As shown in the figures above, the model experiences a "Sudden Leap" in capability. In the final 10% of training time, it acquired the last two-thirds of the problems. This suggests that once a "critical mass" of features is established, learning complex tasks becomes significantly easier.
Knowledge Scaffolding and Feature Reuse
The efficiency gain is attributed to Knowledge Scaffolding. Simple functions act as building blocks for more complex ones. For example, an OR gate detector learned early on can be recruited to help solve a multi-bit adder or a higher-order logic function.
This hierarchical recruitment allows the network to grow in complexity linearly with the number of functions learned, rather than exponentially. The TDL successfully builds deep hierarchies of features, using low-level detectors to feed into sophisticated high-level ones without manual intervention.
Conclusion
The TDL demonstrates that meta-learning through automatic feature recruitment can overcome the limitations of static learning bias. By treating learning as a cumulative process rather than a series of isolated events, the system achieves greater efficiency and structural compression. Future work will address the challenges of "forgetting" obsolete knowledge and scaling these principles to noisier, real-world datasets.
Summary updated