Introduction
In robotics and artificial intelligence, the recognition of spatio-temporal patterns—sequences defined by both spatial location and temporal progression—remains a significant challenge. Tasks such as interpreting hand gestures or recognizing digits drawn in the air require models that can handle time-sensitive information efficiently. Traditional machine learning approaches often demand extensive labeled datasets and typically require the completion of an entire sequence before a classification can be made. For real-time human-robot interaction (HRI), this latency is unacceptable; robots must be capable of recognizing intent early in the action sequence.
This paper explores an unsupervised approach using Spiking Neural Networks (SNNs) to address these issues. By mimicking biological neural processes, the proposed system aims to achieve high classification accuracy with minimal training data and the ability to identify patterns before they are fully executed.
Methodology
The core of the research lies in the application of SNNs, which communicate via discrete pulses or "spikes" rather than continuous numerical values.
Neuron Model and Encoding
The network utilizes the Izhikevich neuron model, incorporating axonal conduction delays to simulate the time signals take to travel between neurons. Input data, such as images or gesture coordinates, are mapped to specific neurons. The temporal sequence of a movement is represented by the order in which these neurons fire. For static images, pixel intensity serves as a proxy for time, with darker pixels triggering earlier spikes.
Unsupervised Learning and PNGs
Learning is achieved through Spike-Timing Dependent Plasticity (STDP), a biological rule where synaptic connections strengthen if a pre-synaptic neuron consistently fires just before a post-synaptic one. This process is entirely unsupervised, requiring no manual labels during training. The network's output is characterized by Polychronous Neural Groups (PNGs)—stereotypical, time-locked firing patterns that serve as unique signatures for different inputs.
Classification
Post-training, the system generates models based on the network's response to training samples. New patterns are classified by comparing their neural signatures to these models using metrics such as the Jaccard index and Longest Common Subsequence (LCS).
Experimental Evaluation
The approach was validated using two primary datasets:
- Hand-drawn Digit Gestures: Video data of humans drawing digits 0-9 in the air, featuring significant variations in scale and translation. Only 7 training samples per digit were used.
- Handwritten Digit Dataset: 16x16 grayscale images where pixel intensity represented temporal order. Only 5 training samples per digit were used.
Results
The SNN approach demonstrated superior performance, particularly in its ability to generalize from very few examples.
As shown in the results, the SNN achieved a success rate of 85.2% on the gesture dataset and 82.5% on the handwritten digit dataset. Crucially, the unsupervised SNN significantly outperformed traditional supervised methods on the gesture task.
While Support Vector Machines (SVM) performed well on the image dataset (86%), their accuracy dropped to 47.36% on the gesture dataset. Other methods like Logistic Regression (LR) and Ensemble Neural Networks (ENN) fared even worse on gestures, highlighting the SNN's robustness to the spatio-temporal complexities of movement data. Furthermore, the system's viability was confirmed through successful implementation on a PR2 robot for real-time interaction.
Conclusion
This research demonstrates that SNNs, utilizing unsupervised STDP and PNG identification, offer a powerful alternative for spatio-temporal pattern recognition. The system's efficiency with minimal data and its capacity for early recognition make it highly suitable for robotics. Future work will focus on transitioning to "on-line learning" to allow robots to adapt during active interaction and scaling the approach to more complex HRI scenarios.
Summary updated