Abstract
This paper introduces CommNets, a decentralized architecture where individual Recurrent Neural Network (RNN) nodes collaboratively solve tasks by sharing their hidden states. Designed for resource-constrained environments like Wireless Sensor Networks (WSNs), CommNets treats internal neural "memories" as a communication medium. This enables distributed event detection without the power and bandwidth overhead of a central processor. We evaluate CommNets on synthetic logical tasks, demonstrating successful convergence and emergent coordination through extremely low-dimensional latent spaces.
Introduction
Wireless Sensor Networks (WSNs) are essential for monitoring diverse environments, from wildlife habitats to clinical settings. However, these networks operate under severe constraints, including limited battery life, minimal memory, and narrow communication bandwidth. Traditional approaches typically involve either centralizing raw data—which is energy-intensive—or processing data locally, which often fails to capture global phenomena requiring multi-sensor correlation.
CommNets addresses this gap by enabling sensors to "talk" to each other using neural network states. The goal is to establish a middle ground where nodes share just enough information to understand global events without overwhelming the network's limited resources.
Methodology: Communicating RNNs
The proposed architecture utilizes Communicating RNNs. Unlike standard RNNs that only propagate information forward through time, CommNets nodes pass information across space to other nodes at every processing step.
Technical Specifications
- Homologous Structure: All nodes share an identical RNN structure and weights. This weight sharing ensures that all sensors "speak the same language," facilitating training and scalability.
- Latent Space Communication: Each node uses only 8 latent states for its hidden representation. This extremely small "message" size minimizes bandwidth consumption.
- Optimization: The model is implemented in PyTorch using ReLU activations. Training is conducted using a combination of Stochastic Gradient Descent (SGD) and the Adam optimizer.
Experimental Setup
The efficacy of CommNets was tested using synthetic datasets modeling three core logical tasks required for multi-sensor event detection:
- Value (T1): An alert triggers if sensor A exceeds a threshold while sensor B remains below another.
- Range (T2): An alert triggers if both sensors fall within specific high or low outlier ranges.
- Compare (T3): An alert triggers if the value of sensor A is greater than sensor B.
Experiments were performed with varying sample sizes ($M$), from 3,000 to 50,000, to evaluate the learning dynamics and convergence of the network.
Results and Discussion
The CommNets architecture demonstrated robust performance across all three tasks. Training loss showed steady convergence, particularly as the number of training samples increased, indicating that the nodes successfully learned to coordinate.
Emergent Coordination
A key finding is the dual role of the hidden state. It serves as both the node's internal memory and the encoded message sent to peers. The results suggest that nodes learn to compress local observations into these 8-dimensional states such that receiving nodes can reconstruct the logic necessary to meet global objectives. This emergent "lingua franca" allows for high efficiency in low-bandwidth hardware.
Conclusion and Future Work
CommNets provides a promising framework for decentralized intelligence in WSNs. By leveraging hidden states for communication, the system achieves complex logical coordination with minimal overhead.
However, several challenges remain. Current tasks are based on clean, synthetic logical gates; real-world sensor data is significantly noisier. Furthermore, the current study assumes a fully connected or simple neighborhood. Future research will focus on scaling the network to hundreds of nodes and investigating the impact of physical distance and intermittent connectivity on hidden state communication.
Summary updated