Abstract
Although necessary, learning to discover new solutions is often long and difficult, even for supposedly simple tasks such as counting. On the other hand, learning by imitation provides a simple way to acquire knowledge by watching other agents do. In order to learn more complex tasks by imitation than mere sequences of actions, a Think Aloud protocol is introduced, with a new neuro-symbolic network. The latter uses time in the same way as in a Time Delay Neural Network, and is added basic first order logic capacities. Tested on a benchmark counting task, learning is very fast, generalization is accurate, whereas there is no initial bias toward counting.
Introduction
Learning to discover solutions for even simple algorithmic tasks like counting is notoriously difficult for machines. While Recurrent Neural Networks (RNNs) and LSTMs can learn to count, they often require tens of thousands of training examples. The core challenge is two-fold: the invisibility of hidden states in standard imitation learning—where a learner sees a teacher’s actions but not their internal "mental" steps—and the requirement for specific structural biases to handle numerical concepts or long-term dependencies.
This paper, originally presented at IJCAI 2007, introduces Think Aloud Imitation Learning (TAIL). The central hypothesis is that if a teacher "thinks aloud"—externalizing their internal state as a sequence of symbols—the learner can imitate these intermediate steps, transforming a complex discovery problem into a straightforward sequence prediction task.
Methods
The proposed system consists of three primary components:
- Think Aloud Protocol: The teacher produces a sequence of symbols including environment states and intermediate "thoughts." The agent "hears" its own output, creating an external recurrence loop that replaces internal hidden states with an observable symbolic stream.
- The Presence Network: A self-growing neuro-symbolic network that replaces fixed-size Time Delay Neural Networks (TDNN). New hidden neurons are created dynamically upon prediction errors, acting as filters for specific sequences. This specificity-based growth allows the network to handle exceptions without degrading general knowledge and provides resistance to catastrophic forgetting.
- Time Delayed Identity Connections (TDIC): This logic layer enables the network to recognize identity between current and past symbols (e.g., "Is the current input the same as the one $k$ steps ago?"). This provides variable binding capabilities similar to first-order logic, facilitating generalization to unseen symbols.
Experimental Results
The system was evaluated on several symbolic benchmarks: the Equality Task, Groups Task (gp), Incrementation, and the $x^n y^n$ formal language.
As shown in Table 1, the performance gains over traditional methods are significant:
- Learning Speed: In the "Groups" task, a standard TDNN (
gp*) required 41,600 sequences to learn, whereas the Presence Network (gp) required only 97.5—a 400x speedup. Similar improvements are seen in theis gptask. - Generalization: On the $x^n y^n$ task, the agent learned from just 13.2 examples on average and generalized perfectly to sequences of length $n=675$ (as seen in the
flwtask metrics), far exceeding the training distribution. - Structural Efficiency: The network remains compact. For the complex $x^n y^n$ task, it utilized approximately 114 neurons (#N) and 3082 weights (#W).
Discussion and Conclusion
By forcing the teacher to "think aloud," the complexity of the learning algorithm shifts from the architecture to the data stream itself. This approach eliminates the need for complex backpropagation through time by externalizing recurrence. While highly effective, the system depends on the teacher's ability to provide a parseable "thought" stream. Future work should explore extending these symbolic methods to continuous domains and scaling to longer-term dependencies.
Summary updated